4 ms·

It’s frustrating that this comment is at the top because it, along with lots of the replies it inspired, absolutely misrepresents the actual declaration. The de

by GPerson 29d ago

It’s frustrating that this comment is at the top because it, along with lots of the replies it inspired, absolutely misrepresents the actual declaration. The declaration is not making any statements about not using any AI in mathematics. The entire point is to push the use of the technology in a direction which is compatible with positive pre-existing features of the math community, and to make it better known what some of the current problems are.

qlte 29d ago
It includes Terrance Tao who has made the front page several dozen times at this point for his usage of AI such as to help the community write proofs for Erdos problems. But to many commentors he's a now gatekeeping AI-hating Luddite clinging to a dying profession out of bitterness and envy because his position is more nuanced than "throw AI at everything and turn off your brain".
- deleted 29d ago
  [deleted]
- zozbot234 28d ago
  Because his letter misuses the word "misalignment" for what's very clearly a capability gap. To anyone familiar with that sort of language, his prose is directly implying that evil superintelligent AIs are deliberately writing obscure proofs in order to harm the community of human mathematicians, which is exactly what the AI-hating Luddite would say! The reality is closer to "current frontier AIs are clueless about what makes mathematical problems/proofs interesting to humans" which is a vastly different issue.
  - necklesspen 28d ago
    LLMs are tools. The responsibility lies in the human(s) using the tool to act accordingly.
  - jhrmnn 28d ago
    It’s actually a fairly precise use of the word, just not in the very specific and narrow sense in which it has been used for AI.
  - freehorse 28d ago
    > his prose is directly implying that evil superintelligent AIs are deliberately writing obscure proofs in order to harm the community of human mathematicians That is probably the wildest misinterpretation of sth I have ever read here. Misalignment is used as in the goals and interests of ai COMPANIES are not the same as the ones of the mathematical community. It is very proper use of the word, and if anything imo the properest, as it refers to actual people and institutions to which actual self-ascribed goals can be defined, as in contrast to hypothetical superintelligence. AI doomers do not own some trademark on the word "alignment".
    - zozbot234 28d ago
      The point stands whether you attribute agency to AIs themselves or to AI companies. The companies are not deliberately sabotaging human mathematical understanding by writing up purposely inscrutable results, either (which is what the word 'misalignment' would imply in this context): they are merely working with frontier tools that have a very limited capability for human-like understanding. It's left to human mathematicians to bridge that particular gap.
      - omnicognate 28d ago
        You seem determined not to understand this, but to spell it out even further: The point is not that the companies are deliberately trying to harm maths but that in pursuing their own goals, which are different to those of mathematicians, they are doing so. Their goals and their methods of achieving them are not aligned with those of mathematicians. This is, incidentally, exactly what alignment means in AI as well: a paperclip maximiser harms humans not because it is intentionally trying to do so but as a side effect of trying to achieve its own objectives, which are not aligned with those of humans (which include not having their entire environment turned into paperclips). The word is being used entirely correctly, and I'm sure the nod to the AI usage is quite deliberate.
        zozbot234 28d ago
        > Their goals and their methods of achieving them are not aligned with those of mathematicians. This is exactly the assumption that Tao is smuggling in with "misalignment" talk and then refusing to elaborate on any further. Is the issue that AI companies are willfully refusing to provide mathematical insight that they could provide (because they have diverging underlying "goals" to those of human mathematicians) or are they merely working under a capability gap, where current AIs can awkwardly settle major open questions but are not smart enough to provide the kind of understanding and insight that the community of human mathematicians relies on? These are two very different problems and by foregrounding the word "misalignment" in his letter so openly (as opposed to talking about AI capability to provide valued insight), Tao is picking the more adversarial reading with zero proof or motivation.
- pred_ 28d ago
  It's the same on Reddit, and if you look up a few comment histories, you'll find that it's mostly /r/singularity users and /r/accelerate posters that are spamming. So I assume that people are just deliberately misreading the letter and trolling here as well, and I wouldn't read too much into it, but on the other hand, if you never had to think about what scientific misconduct looks like, it's harder to see why the behavior of the AI companies is problematic.
  - qingcharles 28d ago
    They're not deliberately misreading it, they are simply just not reading it and just assuming they know what it says.
- vanderZwan 28d ago
  It kind of reminds me of Clay Shirky's predictions about the influence of the internet on society from a few decades ago. To paraphrase what he said: when the printing press was invented, everyone predicted it would lead to world peace, and we got the Thirty Years War instead. When modern mass media was invented people predicted world peace, and instead the Nazis rose to power and we got World War 2. And yet as a society we typically consider these technologies a net positive these days, because alongside all the political instability and violence that followed them at first, some people were figuring out how to do beneficial things with them along the way that did more good than bad. Because the issue really was that the influx of more voices and ideas shifted the power dynamic, required relearning how to communicate, and in the end be better as a society. The issue was figuring out how to "domesticate" these wild new communication channels. One successful example of which was the invention of scientific journals and papers. Shirky predicted the internet would probably lead to a few decades of political instability too (about fifty years was his guess), and we definitely seem to be in the middle of that process. Now, I haven't checked his stance on LLMs. I also don't know if I would quite call them a medium for mass communication like the others (as used today, they take humans out of the loop rather than let more voices join the public discourse), but I feel like they are similar enough to otherwise fit the pattern. Tao's stance similarly feels about wanting to domesticate this wild animal before we get mauled by it. And to stick with the metaphor, I feel that most of the time the AI industry is trying to bamboozle us with spectacular rodeo displays instead.
tmhn2 29d ago
I certainly didn't intend to imply that the declaration is against using AI. My point is just that I feel a little optimistic that the human culture around math ("ideas we disseminate in talks, private discussions and careful writeups, connecting them to the previous ideas of others" and so on, to quote the declaration) is robust even against relatively irresponsible use of AI (i.e., an onslaught of proof-slop). And I guess we'd better try to be optimistic, because even if the declaration results in some realignment between the community and big AI companies, the models capable of this work are not always going to be exclusively under the control of those aligned parties. That said, I think the declaration is great and I support it--let's see what comes out of it.
- eru 29d ago
  > the models capable of this work are not always going to be exclusively under the control of those aligned parties. Yes, the trailing edge will catch up quickly. I wonder how long until P vs NP falls.
voidhorse 28d ago
Yes, thank you. The demise of basic reading comprehension skills will be the true end of us all.
40four 28d ago
I don’t see why the two are mutually exclusive to be honest. I’m not a mathematician, but I’m a software engineer and whether I like it or not, AI has changed my field dramatically forever. I understand the distaste for the way AI might be affecting the field of mathematics, but the genie is out of the bottle, and it will never be the same. It just took a little longer for it to hit the math community. Software engineers have been debating and coming to terms with the new reality for a few years now.
- looofooo0 28d ago
  Soon AI mathematics will be 10x or 100x or 1000x more productive than humans in the proofing theorems business. Mathematics of old style is dead, they should develop ideas how to deal with that as a math community.
  - tehjoker 28d ago
    If a human doesn't understand the proof, it might be useful for certain kinds of engineering, but not really that useful for mathematics except as an existence test.
    - looofooo0 28d ago
      I don't known, can't we just also indirectly observe the wide platonic landscape of mathematics through observing AI results which humanity could not produce in a 1,000,000 years.
      - tehjoker 26d ago
        I imagine this will become one kind of branch of mathematics.
  - GPerson 28d ago
    > But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal.
dgreensp 28d ago
The declaration literally opens by writing that AI companies (or really anybody) saying, “Hey, let’s see if this powerful reasoning engine can solve an open problem in mathematics” is “detrimental” to the “science” of mathematics. Full stop. So I don’t think it’s being misrepresented. If it’s a science, why isn’t progress good in itself? You can’t have it both ways. The online comments I’ve read that side with the letter explain that the problem is AI proofs are inscrutable and useless, but the reality is using computers to brute-force things has long been part of proofs in one way or another, and AI proofs range in legibility up to 100% (like pointing out a proof exists in a long-lost paper, or writing something that is basically correct and just needs a human reviewer to fix it up). People are probably defending the declaration inaccurately, but I think that’s because the real argument or arguments are unclear. Is it that students won’t learn math if there’s ChatGPT? That could be discussed. To a non-mathematician (MIT physics and CS ‘06), the letter sounds like, “We have a fun job. Sometimes there are no practical applications of the work, so it’s just kind of like a sport, er I mean science. If someone solves a problem that has stumped mathematicians for decades or centuries, we worship them as a great mathematician. It’s a status thing. So we don’t like some guy with a computer coming along and solving our problems. The way things unfold with all the ideas coming from humans, over time, maybe it’s slower, but it’s nice.” I don’t think anyone can really stop someone from using a computer capable of solving unsolved problems to solve unsolved problems, and there are always going to be mathematicians who DO think it’s fun to try to figure out what a computer is doing (if the computer isn’t already explaining it in English, which it generally can), and make progress that way, and it obviously will lead to mathematical advances in human understanding, from my point of view. So the whole thing is a non-issue that no one can do anything about anyway.
- bayindirh 28d ago
  Mathematics sits in a very different place w.r.t. other sciences. First of all, it's very meta. Yes, it can describe any phenomenon on our planet. On the other hand, advanced mathematics is a high mountain which is very hard to climb, and changes the climber in various and irreversible ways once the journey begins. That's said, I'm a non-mathematician, but a computer scientist. Mathematics was always my weak part, because solving problems which doesn't mean anything doesn't motivate me, and when I'm not motivated, I fail. Without digressing so much, I want to say that, some of the things in mathematics and mathematics adjacent sciences baffle me. Many mathematicians don't understand the proofs of others, yet they accept it since it checks out within the rules of mathematics. Moreover, many engineers don't understand the formulae they work with. I have developed a very performant Boundary Element Method evaluator, yet I don't know the reason of taking Gaussian Integrals over the surface. The only answer I got is "because the method works that way". So, if computers can make the proofs now, and we have no curious students pecking their professors to understand how these works, the whole mathematics as a science could wither and die. Because besides it being a sport, it's the very language which can model and explain anything, but it's very hard to master and understand due to that nature. Our professors warned us against using too much Mathematica, to make us learn things the hard way and make the knowledge permanent. Now, if we offload science to a matmul engine, it's possible that we lose the connection and never able to close the gap in some cases. Again, it boils down to "This machine has no brain, use yours (or lose it)".
jborekv 28d ago
[dead]

by GPerson 29d ago

qlte 29d ago

It includes Terrance Tao who has made the front page several dozen times at this point for his usage of AI such as to help the community write proofs for Erdos problems. But to many commentors he's a now gatekeeping AI-hating Luddite clinging to a dying profession out of bitterness and envy because his position is more nuanced than "throw AI at everything and turn off your brain".

deleted 29d ago
[deleted]
zozbot234 28d ago
Because his letter misuses the word "misalignment" for what's very clearly a capability gap. To anyone familiar with that sort of language, his prose is directly implying that evil superintelligent AIs are deliberately writing obscure proofs in order to harm the community of human mathematicians, which is exactly what the AI-hating Luddite would say! The reality is closer to "current frontier AIs are clueless about what makes mathematical problems/proofs interesting to humans" which is a vastly different issue.
- necklesspen 28d ago
  LLMs are tools. The responsibility lies in the human(s) using the tool to act accordingly.
- jhrmnn 28d ago
  It’s actually a fairly precise use of the word, just not in the very specific and narrow sense in which it has been used for AI.
- freehorse 28d ago
  > his prose is directly implying that evil superintelligent AIs are deliberately writing obscure proofs in order to harm the community of human mathematicians That is probably the wildest misinterpretation of sth I have ever read here. Misalignment is used as in the goals and interests of ai COMPANIES are not the same as the ones of the mathematical community. It is very proper use of the word, and if anything imo the properest, as it refers to actual people and institutions to which actual self-ascribed goals can be defined, as in contrast to hypothetical superintelligence. AI doomers do not own some trademark on the word "alignment".
  - zozbot234 28d ago
    The point stands whether you attribute agency to AIs themselves or to AI companies. The companies are not deliberately sabotaging human mathematical understanding by writing up purposely inscrutable results, either (which is what the word 'misalignment' would imply in this context): they are merely working with frontier tools that have a very limited capability for human-like understanding. It's left to human mathematicians to bridge that particular gap.
    - omnicognate 28d ago
      You seem determined not to understand this, but to spell it out even further: The point is not that the companies are deliberately trying to harm maths but that in pursuing their own goals, which are different to those of mathematicians, they are doing so. Their goals and their methods of achieving them are not aligned with those of mathematicians. This is, incidentally, exactly what alignment means in AI as well: a paperclip maximiser harms humans not because it is intentionally trying to do so but as a side effect of trying to achieve its own objectives, which are not aligned with those of humans (which include not having their entire environment turned into paperclips). The word is being used entirely correctly, and I'm sure the nod to the AI usage is quite deliberate.
      - zozbot234 28d ago
        > Their goals and their methods of achieving them are not aligned with those of mathematicians. This is exactly the assumption that Tao is smuggling in with "misalignment" talk and then refusing to elaborate on any further. Is the issue that AI companies are willfully refusing to provide mathematical insight that they could provide (because they have diverging underlying "goals" to those of human mathematicians) or are they merely working under a capability gap, where current AIs can awkwardly settle major open questions but are not smart enough to provide the kind of understanding and insight that the community of human mathematicians relies on? These are two very different problems and by foregrounding the word "misalignment" in his letter so openly (as opposed to talking about AI capability to provide valued insight), Tao is picking the more adversarial reading with zero proof or motivation.
pred_ 28d ago
It's the same on Reddit, and if you look up a few comment histories, you'll find that it's mostly /r/singularity users and /r/accelerate posters that are spamming. So I assume that people are just deliberately misreading the letter and trolling here as well, and I wouldn't read too much into it, but on the other hand, if you never had to think about what scientific misconduct looks like, it's harder to see why the behavior of the AI companies is problematic.
- qingcharles 28d ago
  They're not deliberately misreading it, they are simply just not reading it and just assuming they know what it says.
vanderZwan 28d ago
It kind of reminds me of Clay Shirky's predictions about the influence of the internet on society from a few decades ago. To paraphrase what he said: when the printing press was invented, everyone predicted it would lead to world peace, and we got the Thirty Years War instead. When modern mass media was invented people predicted world peace, and instead the Nazis rose to power and we got World War 2. And yet as a society we typically consider these technologies a net positive these days, because alongside all the political instability and violence that followed them at first, some people were figuring out how to do beneficial things with them along the way that did more good than bad. Because the issue really was that the influx of more voices and ideas shifted the power dynamic, required relearning how to communicate, and in the end be better as a society. The issue was figuring out how to "domesticate" these wild new communication channels. One successful example of which was the invention of scientific journals and papers. Shirky predicted the internet would probably lead to a few decades of political instability too (about fifty years was his guess), and we definitely seem to be in the middle of that process. Now, I haven't checked his stance on LLMs. I also don't know if I would quite call them a medium for mass communication like the others (as used today, they take humans out of the loop rather than let more voices join the public discourse), but I feel like they are similar enough to otherwise fit the pattern. Tao's stance similarly feels about wanting to domesticate this wild animal before we get mauled by it. And to stick with the metaphor, I feel that most of the time the AI industry is trying to bamboozle us with spectacular rodeo displays instead.

tmhn2 29d ago

I certainly didn't intend to imply that the declaration is against using AI. My point is just that I feel a little optimistic that the human culture around math ("ideas we disseminate in talks, private discussions and careful writeups, connecting them to the previous ideas of others" and so on, to quote the declaration) is robust even against relatively irresponsible use of AI (i.e., an onslaught of proof-slop). And I guess we'd better try to be optimistic, because even if the declaration results in some realignment between the community and big AI companies, the models capable of this work are not always going to be exclusively under the control of those aligned parties. That said, I think the declaration is great and I support it--let's see what comes out of it.

eru 29d ago
> the models capable of this work are not always going to be exclusively under the control of those aligned parties. Yes, the trailing edge will catch up quickly. I wonder how long until P vs NP falls.

voidhorse 28d ago

Yes, thank you. The demise of basic reading comprehension skills will be the true end of us all.

40four 28d ago

I don’t see why the two are mutually exclusive to be honest. I’m not a mathematician, but I’m a software engineer and whether I like it or not, AI has changed my field dramatically forever. I understand the distaste for the way AI might be affecting the field of mathematics, but the genie is out of the bottle, and it will never be the same. It just took a little longer for it to hit the math community. Software engineers have been debating and coming to terms with the new reality for a few years now.

looofooo0 28d ago
Soon AI mathematics will be 10x or 100x or 1000x more productive than humans in the proofing theorems business. Mathematics of old style is dead, they should develop ideas how to deal with that as a math community.
- tehjoker 28d ago
  If a human doesn't understand the proof, it might be useful for certain kinds of engineering, but not really that useful for mathematics except as an existence test.
  - looofooo0 28d ago
    I don't known, can't we just also indirectly observe the wide platonic landscape of mathematics through observing AI results which humanity could not produce in a 1,000,000 years.
    - tehjoker 26d ago
      I imagine this will become one kind of branch of mathematics.
- GPerson 28d ago
  > But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal.

dgreensp 28d ago

The declaration literally opens by writing that AI companies (or really anybody) saying, “Hey, let’s see if this powerful reasoning engine can solve an open problem in mathematics” is “detrimental” to the “science” of mathematics. Full stop. So I don’t think it’s being misrepresented. If it’s a science, why isn’t progress good in itself? You can’t have it both ways. The online comments I’ve read that side with the letter explain that the problem is AI proofs are inscrutable and useless, but the reality is using computers to brute-force things has long been part of proofs in one way or another, and AI proofs range in legibility up to 100% (like pointing out a proof exists in a long-lost paper, or writing something that is basically correct and just needs a human reviewer to fix it up). People are probably defending the declaration inaccurately, but I think that’s because the real argument or arguments are unclear. Is it that students won’t learn math if there’s ChatGPT? That could be discussed. To a non-mathematician (MIT physics and CS ‘06), the letter sounds like, “We have a fun job. Sometimes there are no practical applications of the work, so it’s just kind of like a sport, er I mean science. If someone solves a problem that has stumped mathematicians for decades or centuries, we worship them as a great mathematician. It’s a status thing. So we don’t like some guy with a computer coming along and solving our problems. The way things unfold with all the ideas coming from humans, over time, maybe it’s slower, but it’s nice.” I don’t think anyone can really stop someone from using a computer capable of solving unsolved problems to solve unsolved problems, and there are always going to be mathematicians who DO think it’s fun to try to figure out what a computer is doing (if the computer isn’t already explaining it in English, which it generally can), and make progress that way, and it obviously will lead to mathematical advances in human understanding, from my point of view. So the whole thing is a non-issue that no one can do anything about anyway.

bayindirh 28d ago
Mathematics sits in a very different place w.r.t. other sciences. First of all, it's very meta. Yes, it can describe any phenomenon on our planet. On the other hand, advanced mathematics is a high mountain which is very hard to climb, and changes the climber in various and irreversible ways once the journey begins. That's said, I'm a non-mathematician, but a computer scientist. Mathematics was always my weak part, because solving problems which doesn't mean anything doesn't motivate me, and when I'm not motivated, I fail. Without digressing so much, I want to say that, some of the things in mathematics and mathematics adjacent sciences baffle me. Many mathematicians don't understand the proofs of others, yet they accept it since it checks out within the rules of mathematics. Moreover, many engineers don't understand the formulae they work with. I have developed a very performant Boundary Element Method evaluator, yet I don't know the reason of taking Gaussian Integrals over the surface. The only answer I got is "because the method works that way". So, if computers can make the proofs now, and we have no curious students pecking their professors to understand how these works, the whole mathematics as a science could wither and die. Because besides it being a sport, it's the very language which can model and explain anything, but it's very hard to master and understand due to that nature. Our professors warned us against using too much Mathematica, to make us learn things the hard way and make the knowledge permanent. Now, if we offload science to a matmul engine, it's possible that we lose the connection and never able to close the gap in some cases. Again, it boils down to "This machine has no brain, use yours (or lose it)".

jborekv 28d ago

[dead]