Small models do the work

When I build with Claude, I generally use Opus to write the plan and audit what comes back. The tasks themselves go to Sonnet agents, running in parallel. I set it up that way because running the expensive model on every subtask costs a fortune. It stays that way because most of those subtasks never needed it.

So when people ask whether niche products still need a frontier model, I already know the shape of my answer. I run a tiny version of the argument every day.

SLMs

NVIDIA Research published a position paper in June 2025, Small Language Models are the Future of Agentic AI. Their definition of small is the useful part: a model that runs on a normal consumer device fast enough to serve one user, which lands somewhere around 3 to 10 billion parameters. Their argument is that agents mostly ask models to do narrow, repetitive things. Calling a tool, formatting output, writing SQL, picking one of four options. Small models are already fine at those, ten to thirty times cheaper to run, and fine-tunable overnight instead of over a quarter.

If you've looked at your own token bill, you already knew this, but what people get wrong is the order.

Bloomberg vs Harvey

The usual claim is that niche companies will win because they own the data. Bloomberg is the cleanest test of that. Forty years of the best proprietary financial corpus anyone has, and in 2023 they spent eight figures training BloombergGPT on it. Then GPT-4 arrived with no financial training whatsoever and beat it across the board, FinQA, ConvFinQA, financial entity recognition, not close on any of them. Four months, roughly, from launch to obsolete. The data was real, it just wasn't the moat everyone assumed.

Harvey is the version that worked, and the sequence is the whole point. It launched running only on OpenAI. In May 2025 it went multi-model and said plainly it would rather fine-tune strong vendor models than train its own, and it built BigLaw Bench so it could tell which model was actually better at what. Only in June 2026, four years in, did it start post-training open-source models that now land close to frontier on legal work. Frontier first, then evals, then routing, then eventually its own models. And the system it runs today still calls GPT-5 when it needs to. Nobody skipped to the end.

Data

The reason the order matters is what "having the data" actually gets you. Mostly it's logs, tickets, contracts, transcripts, a decade of PDFs. That's a corpus, and a corpus is not a training set. You need labels, an eval you trust, and examples of what good output looks like on your specific task.

The thing that produces all three is a big model. It labels your logs, generates your instruction pairs, and sets the accuracy number the small model has to clear. Ethan Mollick made this point right after the Bloomberg comparison landed and it has held up. The frontier model is how you get the fine-tuning data in the first place. It doesn't leave, it just changes jobs, from doing the work to teaching a smaller model to do it and catching what that model drops.

Where I'm less sure

Every fine-tuned small model is a bet that the frontier won't eat your task before you earn the training cost back. Bloomberg lost that bet in a quarter. And the real competitor to your fine-tuned Gemma isn't the giant model, it's a cheap hosted small one from a frontier lab with a decent prompt and no training pipeline to maintain. Sometimes that's just better.

My Sonnet agents aren't fine-tuned on anything. Small, hosted, generic, and they work fine. I have no plan to change that.

The same logic is why Xwor, the voice dictation app I build, runs its models on the device instead of a server. Dictation is a narrow, repetitive task, the exact shape small models are good at. Keeping it local means it's cheap to run, fast enough to serve one person, and nothing anyone says has to leave their laptop. The small model does the work, and the privacy and the cost both come for free with that choice. For diversifying the product, instead of going with APIs of existing LLMs, we can start with models that can run locally and are trained on User's primary and secondary languages, with code-switch understanding.

My claude setup works because Opus wrote a plan good enough that a smaller model can follow it without knowing why. Get the plan wrong and nothing underneath saves you. I still can't tell in advance which tasks are plan-shaped and which need the big model sitting in the loop, so for now I guess, and then I audit.