But isn’t that what “training” is anyway? They train LLM today like that and the database becomes the parameters. You can post train on smaller corpus for purpose-built stuff.
Honestly just give it time. This stuff moves so fast next month the conversation will be different. For folks who don’t like the ID privacy issues, use Deepseek et al and it should be able to get the job done even if the experience takes a bit more wrangling.
The problem with the ID verification is that they can pair introspective conversations with ID. Either that bothers people or it doesn’t.
Main point: we can’t fret about current state models because the ID verification has future implications. Models will change and competition will catch up. Do what feels right in the long run not whether TODAYS model is better at Anthropic.
Both Anthropic and OpenAI don't want to continue training models indefinitely.
Anthropic CEO has expressed potentially slowing down on model training. There is little return for billions of dollars burnt for 1-2% increase on various benchmarks. These companies profit via inference.
Not to mention, the whole Fable being banned by the US Gov is a scary prospect for future models. What is the point of spending billions if its going to get blocked?
Of course this can't go on forever. Especially not on LLMs. But are we really close to the limits of what these LLMs can do? I'm not sure we are.
The difference between GPT-5/Opus 4 and GPT-5.5/Opus 4.8 is striking. For software development anyway, there's no comparison. And all this has happened in a year.
My assumption is there will be another 2-3 years of improvements ahead of us on LLMs alone. Through hardware upgrades, larger training runs, better data quality, better algorithms, etc.
Of course, by then these models will be quite expensive. Will my company pay for it? I don't know. I'm sure some people will though.
The irony is the current administration’s posturing against Chinese AI companies forcing something like this is going to actually bolster competitive advantage overseas.
That and European companies as well. The landscape is going to change drastically in 5 years once all the data centers are built all over the world.
The science behind these models are being worked on IN PUBLIC. The research is not secret. The implementations will all catch up.
> The science behind these models are being worked on IN PUBLIC. The research is not secret. The implementations will all catch up.
Only to a limited extent - the US companies stopped sharing research a long time ago, other than Anthropic's interpretability research (which also seems to have dried up?). Interestingly most of the sharing is now coming from the Chinese side, largely DeepSeek. Ziphu/Z.ai (GLM) is also partner in the Slime RL training framework.
I wouldn't call much, if any, of this "science" - it's all empiricalism. Throw spaghetti at the wall and see what sticks. There's a famous quote from Noam Shazeer:
"We offer no explanation as to why these architectures seem to work; we attribute their success, as all else, to divine benevolence"
Jakob Uszkoreit has also talked about the empiricalism that it took to make what would become the Transformer, and any complex neural network architecture work.
While OpenAI and Anthropic have not provided any useful information for a long time, there still are some research publications from a few US companies, e.g. NVIDIA about its Nemotron models, or Google and IBM about their small LLMs.
