Recalling their recent misuse report [0], I kind of agree with them. And performance/intelligence wise I think we are already at quite a nice local optima.
This reminded me of anecdotes of people discussing with friends about buying a random specific item, and then suddenly seeing it advertised everywhere before even googling about it.
Next step, discussing your Navier Stokes solutions with friends might require leaving your phone in another room.
At least in my experience, the issue with ArXiv is that the expectation is that the draft should be already in a good enough state. And polishing plus writing the meat around the main result can take a lot of time
This is true, as preprints are addressing the 'problem' at a later point in time. I guess git is better if you do not want to assign an identifier to an immutable version yet.
I also think that it's quite a bad PR for them, is it really worth the Millenium prize? Is it not enough that top mathematicians are already actively using these tools? In the long term this would lead to potentially profitable collaborations with universities? Why throw it away so early? Unless they really believe they're gonna solve all math problems now and reputation doesn't matter.
The ultimate drive for some researches is the pursuit of knowledge. If I'm stuck at some block which prevents me from continuing in some direction that I want, of course I would like some help. I believe we already have nonzero collaborative proofs on math.SE, I can't recall good examples, but I have definitely seen citations to mathSE before.
So for me it sounds quite natural to also share this with AI especially under the privacy assumption. Also there's the assumption of scale -- maybe your problem is not large enough for anyone to care to scoop; and just for blind retraining, how do they know that the proof is even correct to include it into training? I have definitely received a ton of incorrect proofs before. So the SNR of such private chats is also not clear. I'm imagining millions of masters/phd students also trying to solve various random things with various capabilities, but how much real signal is there?
Maybe a dumb observation, but if a chain of people were working on the problem for a long time, it's not difficult to imagine that someone accidentally prompted a model with their personal or some other account without the privacy set correctly.
Then again, maybe this is my internal cope, hoping that they're not secretly training on private chats.
[0]: https://www.anthropic.com/threat-intelligence-report-septemb...
reply