> but it is undoubtedly and objectively accelerating research.
Part of the point of the letter is that it is quite possible to act in a way that is a net negative to research. The most obvious case is when the companies violate ethical standards in research.
The subtler case, the one for maths in particular, is what happens when you fail to follow well-established patterns for making maths research productive. Tao himself spelled out how that can look in https://mathstodon.xyz/@tao/117207856734787448 (which notably came before any of the news on Navier–Stokes).
> The declaration literally opens by writing that AI companies (or really anybody) saying, “Hey, let’s see if this powerful reasoning engine can solve an open problem in mathematics” is “detrimental” to the “science” of mathematics. Full stop.
So let's just quickly agree that the actual quote is “However, the push by AI companies to solve mathematical problems as a benchmark is detrimental to the science of mathematics, and to the mathematical community.” And that in this, "as a benchmark" is load-bearing.
We wouldn't see nearly the same amount of contempt from researchers had OpenAI picked a research-friendly approach.
What they did: Hear a rumour about the problem being solved by other researchers, then rush to scoop them (unethical), then, when they actually go talk to them, they try to oust an author (also unethical), and when they finally decide to share their own work, do so in the least useful way possible.
What they could have done: Upon hearing the rumours, connect with the researcher in question and propose that they join efforts instead; set up a joint project to test if the machines are useful in any way, and if that's not appreciated, back down again. And instead of dumping only an undigested paper* and a Lean proof, do the digestion prior to publishing anything (as Buckmaster was in the process of doing). If their own lack of competences was keeping them from digesting it, then again, reach out to the researchers to understand if anyone would be willing to do so.
In the second of those two worlds, we wouldn't be seeing nearly the amount of outrage that we are seeing right now.
*: Here, “digestion” is the process of turning an AI slop paper into something humans can read. LLMs can indeed sometimes (if much more rarely than marketing material from the large LLM companies will suggest) produce correct proofs, but they are often written in bizarre ways – they'll use lingo that doesn't exist, seem overly pretentious, dwell on extremely easy steps while glossing over the hard ones. Currently, a real researcher will take that output and transform it into something that others can understand, use, and build upon. This is not so different from what happens when using it to write software, although as someone who does both, I will say that the amount of digestion needed for proofs tends to be orders of magnitudes larger than for code. This meme is quite accurate: https://mathstodon.xyz/@tao/117068266071803252
There are several cases of this already. Bubeck himself had to retract earlier claims of novelty, and more recently, the provenance of the non-sofic group result was brought into question. Most recently, it turn out that the construction used for Anthropic's counterexample to the Jacobian Conjecture had appeared in an unpublished but publically available draft: https://news.ycombinator.com/item?id=49657499
This has a few practical implications: First of all, if you are in the target group of the marketing material, be wary. While these things can do non-trivial stuff, the amount of magic is being grossly over-stated. But also, when several of the big results have indeed been reappropriating the work of others; when the companies fail to provide proper attribution (the NS case in particular is laughable) and present the results as the models' own work, that's plagiarism.
And just to spell it out, since it looks like HackerNews is flooded by people who are new to science these days: even if a result doesn't come with a price, scholarly peer review is the norm across all of science: https://en.wikipedia.org/wiki/Scholarly_peer_review
And chances are they never will publish it in any kind of useful format. Right now, the scientific community is outraged at OpenAI for going about their announcement in the least productive fashion they could have. It really does seem like they have no interest in progressing our understanding of maths outside of mining it for marketing material.
Boo hoo. OpenAI got the result only days ago. It makes perfect sense for them to take the win in marketing, and it's fine if they take a few months putting together the paper and present it more productively later. The scientific community didn't get the result themselves, so it isn't theirs to be bossing everyone else around about.
I don't care much for AI myself, or smart phones either, for that matter. I would be content if NS remained a mystery for another 100 years - or forever. But goodness, does the "scientific community" need to take a deep breath and count down from 10.
Was it a marketing win though? My takeaway is: if you're doing groundbreaking work with openAI's models and they find out, at best they'll outspend you and scoop you. At worst they'll steal your chat history.
That's a completely different matter. And does it matter to my point if they were successful or not? That's ex-post analysis. It seems clear that ex-ante, they wanted this to be a marketing win. The original poster complained that they wanted a marketing win.
My point is: why shouldn't they want a marketing win from this. What obligation does a non-academic institution have to follow the traditions of academia? Its result doesn't belong to academia. And if academia wants to subject OpenAI to their own internal processes and give them marching orders, it just isn't going to work and maybe - who knows - it'll even further erode their own legitimacy. Does anyone actually believe that NS would have been resolved in the 2020's if we lived in a parallel world where LLM's were never invented? Would Buckmaster have gotten as far as he did without LLM's doing a lot of the work for him? We can complain about AI companies contributing to mathematics, but are we complaining about Terence Tao using AI in his research? When Tao publishes something are we all going to go to war against him because maybe other mathematicians' prompts went into training the AI that Tao used?
It's the same on Reddit, and if you look up a few comment histories, you'll find that it's mostly /r/singularity users and /r/accelerate posters that are spamming. So I assume that people are just deliberately misreading the letter and trolling here as well, and I wouldn't read too much into it, but on the other hand, if you never had to think about what scientific misconduct looks like, it's harder to see why the behavior of the AI companies is problematic.
> To be honest, I feel like the difficulty of reading AI proofs is due to the fact that we are on the verge of being beyond human comprehension.
I can see where that's coming from, but I really don't think it's the case. Even with Astra, the proofs you get are just off in a way that doesn't signal superhuman comprehension. As 9question1 says, a common theme is that they dwell on insignificant steps. Another one is that they'll often be full of terminology that either doesn't exist, or has this weird quality where it looks like it is trying to make some minor insight seem much greater than it is. At first glance, that'll often make it look like it knows more than you, but when it's really just doing the same thing but in a more complicated and worse fashion, that to me isn't a signal of comprehension at all. The bizarre thing is that despite all the "stochastic parrot" style nonsense you'll get in individual proof steps, they still often combine to something valid.
In either case, what all of this means is that the working mathematician still needs to go through, and generally completely rewrite, any proof output by an LLM. Otherwise you are passing the burden of unreadability onto the reader.
Yeah, that mirrors what I've seen throwing some of the leading models at a set-theory problem that's stumped me (https://mathoverflow.net/q/511601): in this case, the problem does not easily yield to the standard tools, but the LLMs do not recognize it as a major open problem they should give up on. So they seriously try it, but typically end up in a loop of inventing certain classes of simple solution or counterexample attempts, defeating them, and trumpeting each one as a major result, each time inventing some new terminology.
It's definitely quite curious that the AI labs are able to push these results through seemingly with pure brute force. Perhaps it's largely a function of how many monkeys you have attempting various constructions on top of the known results and strategies the models have memorized.
If frontier labs had chosen to go the path of offering to assist in existing endeavors, helping to build knowledge alongside researchers in ongoing projects and following ethical and professional research standards, we wouldn't be having this discussion at all; everyone would be stoked. Instead we have companies that disgracefully try to scoop researchers and fail to properly attribute earlier work and instead rebrand it as their own (what we normally call plagiarism) to make marketing material.
Tao doesn't go as far as Baudelaire, but there are some similarities. In particular, Tao has criticized that AI is not being used to create new interesting conjectures, and that the rush to prove old conjectures is not giving human mathematicians enough time to carefully analyze and understand the proofs and the methods used in those proofs.
My answer to both is the same: nothing stops mathematicians from doing both of those things, with or without the help of AI. And we all understand that it will take time to do that. But complaining about the dawn of a new era of advancements seems counterproductive.
There are no similarities whatsoever, other than them having opinions on a new tech that affects their field.
> nothing stops mathematicians from doing both of those things
Actually yes, and they explain that pretty clearly in the letter. These problems have been used as "goals" not because the solutions are necessarily worth it, but because they set a clear direction whose process would probably yield very useful theories, and also more conjectures and problems. If the goals get solved by AI models that do not generate comprehension, then the direction gets less clear, it's harder to collaborate and get funding (think what would be easier, to get funding to try and solve Navier-Stokes or to try and understand an already proven Navier-Stokes?), and we get less open problems to work on.
It all comes down to this. Yes, they will need to give a good and convincing argument to the taxpayer. And at the end of the day, if the circumstances change, and you're unsuccessful in making your case in your plead, the taxpayer and their representatives may not want to keep the same level of funding and instead put that money to use elsewhere. So make a good argument why your job is still worth that cash under the new circumstances. The argument can appeal to aesthetics and art and philosophy and a kind of feeling of religious elegance and so on, it's fine. I can't say how well that will work, but it's worth a try. Taxes also fund theology and other pursuits with no tie to cold hard engineering etc. But that level of funding is not the same as for math currently. That gap may shrink.
I don’t get the argument you are making. If you want to reduce funding for mathematics research, argue for that. But wanting to make getting funding and coordination and collaboration harder in exchange for nothing other than PR for AI labs does not make sense
Is it just PR? Okay then the useful math can be done by AI and we can leave aesthetic math to human academics who will ascetically refrain from using AI and if anyone of them submits a paper and it turns out they used computers they will be treated like WADA treats Olympians on steroids or chess players with a vibrator up their ass telling them what move to take. But again the taxpayer may not foot the bill for this endeavor as much as they did while thinking it may be useful for technology, science and engineering innovation.
But did you read the letter, or are you familiar with the people signing it? Several things tell me you didn’t:
- The useful part of math is the tools, the understanding and the theories. Knowing whether the result is true or not is usually not useful. Imagine an AI that just says “cancer can be cured” but doesn’t say how: that’s what the letter is criticizing.
- Terry Tao is one of the biggest proponents of AI use in math. He’s been for quite some time and he shares a lot on his blog.
Seriously, it seems like you’re arguing about some idea you have in your head and you’re not making the effort to actually understand anything about this letter.
AI won't just say cancer can be cured without saying how. Not sure where you got this from. AI will be superhuman at writing the executive summary at any prerequisite knowledge level.
These people are concerned about internal community dynamics like tenure, chairs, PhDs, grad students, the whole ecosystem, papers, hiring, conferences, journals etc. But the wider population doesn't care about this and academics don't understand that nobody cares about this outside their bubble.
As Feynman said, "mathematicians can research what they please. If you want something else, you work it out yourself" (not verbatim). Indeed, now people can do this with AI and it makes mathematicians salty that they will have less prestige.
> AI won't just say cancer can be cured without saying how.
It was an example, an analogy to argue that answering a question might not be as important as how it arrives to that question. Hence the “imagine”.
> These people are concerned about internal community dynamics like tenure, chairs, PhDs, grad students, the whole ecosystem, papers, hiring, conferences, journals etc.
You just described most of the parts of the mathematical community.
> But the wider population doesn't care about this and academics don't understand that nobody cares about this outside their bubble.
I don’t care about the logistic system and yet I want my food to be in the market. If the population at large wants to use mathematics they will have to find a way to support a mathematical community. Now, if you don’t want mathematics, again, go argue about that.
> Indeed, now people can do this with AI and it makes mathematicians salty that they will have less prestige.
Except that they can’t. I do not know how many more times we need to explain that “answering questions” is not what mathematics is about. That if that path is followed, then mathematics will suffer as a result and the pace of mathematical development will slow down.
Honestly, I think it would be better for you to really stop attributing the arguments of mathematicians that are by no means anti AI to saltiness or other childish feelings. Try to be more humble and try to find what kind of ideas they might hold for them to argue this letter. Of course, that requires reading the letter and trying to understand it.
> You just described most of the parts of the mathematical community
Just as science (as in the scientific method) isn't the same as Science, the community social practice and logistics and tenure rules and committee compositions and journal page limits etc, mathematics is not the same as the academic mathematics community. They are not the sole producers of mathematics and not the sole users. The funding is largely in the hopes of the usefulness of the resulting math. Not all. Some funding is like art and culture funding, for propagating a cultural legacy, like folk dance also gets funding and experimental performance theater also gets some tax funding. But that's not the bulk currently in math.
I have read the letter. It's all about stuff of holding back because they don't want the answer key because prestige reasons, and how will grad students train their brain if we have too many answers. This doesn't consider that many people, like engineers, use math as a tool. They don't want to hold math back. It's fine to think that applications are too dirty. Again, they want funding, they have to explain why exactly they should get it. And it better be an explanation that still holds water with AI available. They can do the aesthetic math as an art side project like an accountant can paint or sing off the clock. But generally you also don't pay everyone for their hobbies even if those hobbies are nice culturally rich endeavors.
> they don't want the answer key because prestige reasons
No, that’s not the reason. They don’t want the answer key because the answer key is useless.
> This doesn't consider that many people, like engineers, use math as a tool
Of course it considers it. Many people, like engineers, could not care less about the answers to most problems mathematicians work on. They do care about the tools they develop in the process. I’ve already shown examples of this but I’ll do it again: Galois theory was developed when trying to answer whether there are formulas to solve roots of polynomials of degree 5. Fourier analysis was developed when trying to find an analytical solution to the heat equation. Riemann developed his geometry to explore which Euclidean axioms were actually important. None of the direct answers were as important as how they got to them.
> Again, they want funding, they have to explain why exactly they should get it. And it better be an explanation that still holds water with AI available.
The explanation will be exactly the same. Only it will not be just hard to explain why the problem is important, but also it will be harder to explain that no, just the answer by itself is not useful without the understanding. Just like I am here having a really hard time explaining to someone who doesn’t understand how mathematical research works why “just getting the answer” is not a useful output.
This hangs on the strange assumption that the AI will "just give the final answer" and will be unable to give a human-preferred explanation. I find this assumption strange. If it holds, then mathematicians will still have a job digesting these proofs. If the AI can digest it better, then academic mathematicians will become humanities professors funded similarly to theology, philosophy and dance professors etc. for the sake of preserving a cultural practice, like traditional handmade broom makers etc.
Why is it strange? Are you familiar with how unsupervised proofs are made? It’s basically iterating on Lean code, even if it were able to explain the code to humans it doesn’t mean it can extract actual understanding and structure from it. The papers being outputted by these approaches are nowhere close to being understandable, much less useful.
> If the AI can digest it better, then academic mathematicians will become humanities professors
Cool, then once that happens we can discuss what to do. In the meantime, AI does not digest proofs properly, does not explain them and does not generate any understanding, and it doesn’t look like that’s going to change. Hence the letter and the criticism made to the approach taken by AI labs. It is not that hard to understand.
> But complaining about the dawn of a new era of advancements seems counterproductive.
you completely misunderstood the critics. your analogy is awful. this is much closer to the industrial revolution in the uk: it brought a lot of progress, but also extreme inequality and concentration of power.
I don't see any meaningful analogy with your cites of Baudelaire. First of all, Baudelaire discusses art, which is very different to science. Several new forms of art have emerged, and then slowly integrated, despite the strong opinions of some. Here, Tao's letter discusses mainly practical aspect of scientific research. It is not about what a "failed mathematician" would do, or if "LLM can only record what already existed"; in fact, it ackowledges that LLM could become a central tool. If you want me to be even more precise, what they're saying (without saying it out loud) is that tech companies have too much power and are being irrespondible with it because they don't even think about the externalities.
> It is not about what a "failed mathematician" would do, or if "LLM can only record what already existed"
This is what I was alluding to:
I wrote recently about how the collection of good, fruitful open problems is now being mined in a non-renewable fashion, leading to the potential scenario of these problems becoming scarce [0]
Of course there's a meaningful analogy. The poster wants to make Tao sound unreasonable and against "progress" while ignoring the very real concerns Tao has.
It's what fanatics do when they want to enforce their view on the world, they have to attack anyone with a reasonable viewpoint because they can't imagine a world where someone tells them they don't like what they're doing.
I do not want to make him sound unreasonable. He is a brilliant man. But I also think that he is a bit shell shocked, much like other luminaries in the past have been when confronted with rapid technological development, so I wanted to highlight some historical similarities, imperfect as they can only be.
As for the rest of your comment, I hope you have a great day.
He’s not shell-shocked. He’s had an extremely consistent narrative for years now. You are experiencing a bias where you project your own views onto others, probably because you are not very informed on this.
Tao is quite the opposite of what you describe. He's had an ongoing dialogue intellectually with the challenges created by ai for mathematicians for some time (years) now and his viewpoint is quite balanced. He is not alarmist but rather correctly addresses the real problems ai advances create for mathematicians traditional way of working.
> But I also think that he is a bit shell shocked, much like other luminaries in the past have been when confronted with rapid technological development
You're locked into that idea, but provide no real argument. The dangers they describe in the letter are real.
It's not about whether the conjectures themselves are interesting or not, or whether people simply have enough time. It's that a bare proof made by a machine doesn't actually do much for us. There is not some set of problems that, once finished, will amount to some kind of final, correct system and we can call it a day and, like, utilize it. "Mathematics" is the people doing it (the "mathematical community" Tao references below). This other stuff is kinda just.. expensive exercises to render a result. They are only actually beneficial to us insofar as they exist in a context of research among peers.
You have expressed a few different ideas, so I will address them separately.
> It's not about whether the conjectures themselves are interesting or not, or whether people simply have enough time.
Tao seems to think otherwise, if I am reading him correctly:
I wrote recently about how the collection of good, fruitful open problems is now being mined in a non-renewable fashion, leading to the potential scenario of these problems becoming scarce [0]
Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others. As in all creative professions, this raises severe attribution and plagiarism questions. [1]
> It's that a bare proof made by a machine doesn't actually do much for us.
I get it, and I think the same can be said about all sorts of human endeavors.
> There is not some set of problems that, once finished, will amount to some kind of final, correct system and we can call it a day and, like, utilize it.
Sure. Although there are certainly practical applications to be found along the way. E.g. proving P=NP would be potentially very significant in the real world. I think we agree.
> "Mathematics" is the people doing it (the "mathematical community" Tao references below)
Sure. And the same can be said again about all sort of human endeavors. But I don't see how that is a reason to stop using AI in those fields, either. It doesn't subtract anything, in the same way that chess engines didn't destroy the love of the game for chess.
And just like in chess, these AIs can be used to gain a deeper understanding. Including, but not limited to, explaining to humans the proof they just came up with.
Are you trying to argue toward some final verdict with regard to LLMs and mathematics? I thought you were just trying to make a comparison to Baudelaire? I think maybe being clearer on this point would help. Even if you argued sufficiently for the latter (which is going to be tough already), it wouldn't really speak to the former. Or at least: that would have to be a separate argument I think.
Also, how, in your words, do you feel like the first quote justifies your point (presumably with regard to the question of "interesting" or not)? And why do you think the second one is more about time itself rather than attribution? Do these things actually contradict the letter above (or the comment on it) in your mind or not?
In general, do you disagree with something here specifically? Or is it kind of a yes/and thing? Does any of this help, in your mind, with the Baudelaire comparison you were at least at one point trying to argue for? Its a bit hard for me to see the argument here, if there is one, just with what you have written. But I am sure I am just not knowledgeable enough to grasp the argument!
My opinion is that the field will adapt. AI will not spell doom for mathematics. On the contrary, it is the beginning of a new era. Math is having its Deep Fritz moment, just as computer science is going through the same.
Some folks are struggling to adapt to this change. Tao actually sounds like he is doing alright compared to most, even if some of his arguments seem a bit weak, as I alluded to in other comments in this thread.
Hopefully it makes some sense. And if it doesn't, at least we had a nice chat.
Haha yes I imagine we all know your opinion is something like that! But isn't it much more fruitful and interesting to form an argument for it? If he is still in the context here: is there something about Baudelaire's critique you were discussing originally that has helped you arrive at this opinion? Do you think he is wrong? Or is he right, but providing the right kind of nuance that we need in order to cope with the revolutionary changes? Genuinely curious!
> But isn't it much more fruitful and interesting to form an argument for it? If he is still in the context here: is there something about Baudelaire's critique you were discussing originally that has helped you arrive at this opinion?
The historical record. That's why I am drawing some lose parallels with Badulaire. Incumbents being unhappy about a disruptive technology, lashing against the early adopters, and fearing that it signifies the end of their craft, when in reality it's just a period of change and adaptation. Without the advent of photography we would not have Impressionism nor all the movements through the 20th century. Photography forced painters to reinvent themselves, and LLMs will force mathematicians to do the same.
I thought the historical examples of photography and chess engines would be enough for people to connect the dots, but apparently not.
> I thought the historical examples of photography and chess engines would be enough for people to connect the dots, but apparently not.
You're just an overconfident idiot who thinks that because there is an analogy along one dimension of a absurdly complex problem you've somehow arrived at a superior truth.
I repeat, this is an incredibly complex situation in which no one fully understands what's going on. The letter is simply asking for greater awareness of the implications of what AI companies are doing, they're not even saying that AI is bad!
Sure, but then it's kinda less that you're making an argument here one way or another, and more of just an appeal, right? Like its just broad strokes? The inherent actual qualities of photography aren't supposed to parallel anything with the LLM case other than just the bare fact of being a new technology at one point?
I guess I am sorry for pushing, I felt like maybe there was going to be something more specific to the point.
I think one thing to keep in mind in the future is that for every Baudelaire, there is a Benjamin or Italian futurist saying the exact opposite! Its not like the fear of photography was a monolithic, shared thing, even at the time. Just, it's not so straightforward to have the "historical record" speak exactly one narrative, and unless you are a full-on technological determinist (is anyone these days?), you can't really conclude anything from the past like this.
But the point is your original analogy with Baudelaire now just boils down to "both of them were critical" which is quite shallow indeed.
It's especially annoying given that many people are making repeat Baudelaire's critique of photography (or Socrates critique of writing) in the context of AI and the declaration is interest because it's not that.
Part of the point of the letter is that it is quite possible to act in a way that is a net negative to research. The most obvious case is when the companies violate ethical standards in research.
The subtler case, the one for maths in particular, is what happens when you fail to follow well-established patterns for making maths research productive. Tao himself spelled out how that can look in https://mathstodon.xyz/@tao/117207856734787448 (which notably came before any of the news on Navier–Stokes).
reply