6 ms·

The whole point of the letter was that if we adopt this gradient search strategy the humanity might get stuck in a local maximum forever.

by varjag 16d ago

bananaflag 16d ago
Thing is, you can say that about any technology (not being here necessarily pro AI in math, but I think we need better arguments).
- varjag 16d ago
  I mean sure? It's clear PLT and other applied CS fields are now as good as dead. You're never seeing new React or Python emerging ever again. And unfortunately mathematics is much more fundamental to human endeavor than this.
  - sampullman 16d ago
    I don't think that's true. Domain experts with the desire to create and experiment aren't going away any time soon.
    - varjag 16d ago
      Sure we are free to tinker, like the guys doing model railways. With about the same impact.
      - sampullman 16d ago
        We'll have to check back in 5 or 10 years to see if anything new has been created.
    - irthomasthomas 16d ago
      It is getting a lot harder for those people to justify using OpenAI to assist such endeavours. Afterall, OpenAI might just front-run you if they hear a rumor you solved some marquee problem that they can brag about in PR campaigns.
- GPerson 16d ago
  The letter is not incompatible with “pro AI in math” unless one thinks arguing against extreme behavior such as corporations using millions of dollars to scoop results makes one “anti AI”, which is not a reasonable stance in my opinion. > AI offers the potential of enhancing and accelerating genuine mathematical study and understanding. Mathematics as a profession will need to adapt to these changes in several ways. However, whether these changes ultimately benefit the field or have a destructive effect will in large part be determined by the decisions of the humans in control of this new technology.
- topaz0 15d ago
  You can say it about becoming collectively overdependent on any technology. And I think there's a good case to be made that a similar phenomenon is true for other technologies e.g. over-reliance on (normal) computers has done similar things in my opinion, at least in my field of physics, where you get a more precise result but much less insight.
Chance-Device 16d ago
Then we develop ways of breaking out of local maxima. Wherever we can recognise a problem we can solve it.
- varjag 16d ago
  Nobody is going to recognize anything if the institutions of mathematics are torn down.
- mattmanser 16d ago
  Tao's point is that we have a method already.
- SideQuark 16d ago
  This isn’t true, as there are plenty of things proven beyond reach. And plenty of things, eventually solvable, can create major problems that could both be avoided and the problem solved by taking a much better path. Having technology and the ability to safely and sanely use the technology needs to progress together at a similar rate. The failure to do this is even a reasonable and common solution to the Great Filter. Jared Diamonds book “Collapse” has ample examples of cultures that wiped themselves completely out via not having this balance, so it’s not simply a theory.
simianwords 16d ago
This pattern of argument keeps repeating in every place. Pro tech people: technology removes bottlenecks. Sometimes we use those bottlenecks as a side effect to build muscle and so on. But removing bottlenecks gives us much higher degrees of freedom. It is up to us to coordinate and make use of the technology. Anti tech people: bottlenecks are fundamentally useful. They should remain and technology shouldn't remove those so easily. Humans cannot coordinate as well when the bottlenecks are removed, so lets not remove them so quickly.
- varjag 16d ago
  I can only suggest reading the letter with open mind and literally, without uncharitable bias towards the authors or conspiracy mindset.
  - simianwords 16d ago
    Its not a conspiracy. Its 25 fields medalists who think exactly like how I put it. Ideally, they can just take whatever the technology gives as soon as possible and use it. They don't want it because they don't think the community can rearrange and coordinate such that they can make use of the new found degrees of freedom. That's literally all there is to it - they don't believe in the rearrangement.
    - varjag 16d ago
      I can only envy the insight you have into thought processes of 25 fields medalists. Wish I could have that.
      - simianwords 16d ago
        you must have been so smug writing this. But I wanted to engage in a discussion.
        varjag 16d ago
        From your other replies it's clear you still haven't read the letter. There is little to discuss so far.
nkrisc 16d ago
“Forever” is a long time. That’s quite the claim.
teekert 16d ago
I think it is easy to just roll of a local maximum, however, you may get stuck in a local minimum. I kid, of course, but I do wonder where the use of local "maximum" comes from, what is maximum there? Why do you not see this as a landscape of hills and valleys where marbles with certain energies may indeed get stuck in deep enough holes... Of course, I just assume and picture gravity pointing down in that landscape, but hey. I'm human, I feel it is expected of me.
- NonHyloMorph 15d ago
  There's some interesting stuff in a book on Katathymic Imaginative Psychotherapy where patients are asked to picture a mountain. As far as I remember it's about using the process as diagnostic tool. What does the mountain look like? Is it a steep rising, rugged massif or a shallow hill. Are you looking up, do you picture yourself climbing it, or are you on it's top etc. While it said that it's always a contextual matter and in the conversation there was a suggestion of imagining to climb the steep massif, being potentially related to narcisim. Whenever I think of it i associatr that with casper david friedrichs painting.
skywal_l 16d ago
Can't we get stuck in local maxima because of the lack of LLMs?
- GPerson 16d ago
  The letter is not advocating not to use LLMs. It’s advocating to use them in ways which don’t erode human understanding.
- RugnirViking 16d ago
  as a society of researchers we've tended to cultivate pretty effective strategies for escaping local maxima. I think of it like ants, where you can see if you place an obstacle in between their nest and a foot source, they develop a path that loops around it. if you remove the obstacle, for some time they continue to follow the old looped path. However, some ants deviate and go around at random, exploring. eventually by chance one happens to find a quicker route. he gets a couple of his friends to follow him, by pheremone, and over time more and more take the quicker route, and they end up abandoning the old route It works this way with research, with most following the current trends, and some curious souls searching around for other ideas, be they contrarians, dreamers, or just convinced of some strange truth. But if we're right, signs tend to slowly begin to point their way, and we can shift the whole hulking edifice of science towards their point of view. The problem of llms is that while they may be able to find a shorter route, we can't follow them unless we understand the route. So the forces that slowly begin to change everyone's behavior are lost
CamperBob2 15d ago
The way you get out of a local maximum is to force yourself to try something radically new every so often, even if what you were doing before was working just fine. This is the 'stochastic' part of 'stochastic gradient descent,' and it's as important for human minds as it is for ANNs. These math wizards seem to be stuck in a rut of their own digging.

The whole point of the letter was that if we adopt this gradient search strategy the humanity might get stuck in a local maximum forever.

by varjag 16d ago

The whole point of the letter was that if we adopt this gradient search strategy the humanity might get stuck in a local maximum forever.

bananaflag 16d ago

Thing is, you can say that about any technology (not being here necessarily pro AI in math, but I think we need better arguments).

varjag 16d ago
I mean sure? It's clear PLT and other applied CS fields are now as good as dead. You're never seeing new React or Python emerging ever again. And unfortunately mathematics is much more fundamental to human endeavor than this.
- sampullman 16d ago
  I don't think that's true. Domain experts with the desire to create and experiment aren't going away any time soon.
  - varjag 16d ago
    Sure we are free to tinker, like the guys doing model railways. With about the same impact.
    - sampullman 16d ago
      We'll have to check back in 5 or 10 years to see if anything new has been created.
  - irthomasthomas 16d ago
    It is getting a lot harder for those people to justify using OpenAI to assist such endeavours. Afterall, OpenAI might just front-run you if they hear a rumor you solved some marquee problem that they can brag about in PR campaigns.
GPerson 16d ago
The letter is not incompatible with “pro AI in math” unless one thinks arguing against extreme behavior such as corporations using millions of dollars to scoop results makes one “anti AI”, which is not a reasonable stance in my opinion. > AI offers the potential of enhancing and accelerating genuine mathematical study and understanding. Mathematics as a profession will need to adapt to these changes in several ways. However, whether these changes ultimately benefit the field or have a destructive effect will in large part be determined by the decisions of the humans in control of this new technology.
topaz0 15d ago
You can say it about becoming collectively overdependent on any technology. And I think there's a good case to be made that a similar phenomenon is true for other technologies e.g. over-reliance on (normal) computers has done similar things in my opinion, at least in my field of physics, where you get a more precise result but much less insight.

Chance-Device 16d ago

Then we develop ways of breaking out of local maxima. Wherever we can recognise a problem we can solve it.

varjag 16d ago
Nobody is going to recognize anything if the institutions of mathematics are torn down.
mattmanser 16d ago
Tao's point is that we have a method already.
SideQuark 16d ago
This isn’t true, as there are plenty of things proven beyond reach. And plenty of things, eventually solvable, can create major problems that could both be avoided and the problem solved by taking a much better path. Having technology and the ability to safely and sanely use the technology needs to progress together at a similar rate. The failure to do this is even a reasonable and common solution to the Great Filter. Jared Diamonds book “Collapse” has ample examples of cultures that wiped themselves completely out via not having this balance, so it’s not simply a theory.

simianwords 16d ago

This pattern of argument keeps repeating in every place. Pro tech people: technology removes bottlenecks. Sometimes we use those bottlenecks as a side effect to build muscle and so on. But removing bottlenecks gives us much higher degrees of freedom. It is up to us to coordinate and make use of the technology. Anti tech people: bottlenecks are fundamentally useful. They should remain and technology shouldn't remove those so easily. Humans cannot coordinate as well when the bottlenecks are removed, so lets not remove them so quickly.

varjag 16d ago
I can only suggest reading the letter with open mind and literally, without uncharitable bias towards the authors or conspiracy mindset.
- simianwords 16d ago
  Its not a conspiracy. Its 25 fields medalists who think exactly like how I put it. Ideally, they can just take whatever the technology gives as soon as possible and use it. They don't want it because they don't think the community can rearrange and coordinate such that they can make use of the new found degrees of freedom. That's literally all there is to it - they don't believe in the rearrangement.
  - varjag 16d ago
    I can only envy the insight you have into thought processes of 25 fields medalists. Wish I could have that.
    - simianwords 16d ago
      you must have been so smug writing this. But I wanted to engage in a discussion.
      - varjag 16d ago
        From your other replies it's clear you still haven't read the letter. There is little to discuss so far.

nkrisc 16d ago

“Forever” is a long time. That’s quite the claim.

teekert 16d ago

I think it is easy to just roll of a local maximum, however, you may get stuck in a local minimum. I kid, of course, but I do wonder where the use of local "maximum" comes from, what is maximum there? Why do you not see this as a landscape of hills and valleys where marbles with certain energies may indeed get stuck in deep enough holes... Of course, I just assume and picture gravity pointing down in that landscape, but hey. I'm human, I feel it is expected of me.

NonHyloMorph 15d ago
There's some interesting stuff in a book on Katathymic Imaginative Psychotherapy where patients are asked to picture a mountain. As far as I remember it's about using the process as diagnostic tool. What does the mountain look like? Is it a steep rising, rugged massif or a shallow hill. Are you looking up, do you picture yourself climbing it, or are you on it's top etc. While it said that it's always a contextual matter and in the conversation there was a suggestion of imagining to climb the steep massif, being potentially related to narcisim. Whenever I think of it i associatr that with casper david friedrichs painting.

skywal_l 16d ago

Can't we get stuck in local maxima because of the lack of LLMs?

GPerson 16d ago
The letter is not advocating not to use LLMs. It’s advocating to use them in ways which don’t erode human understanding.
RugnirViking 16d ago
as a society of researchers we've tended to cultivate pretty effective strategies for escaping local maxima. I think of it like ants, where you can see if you place an obstacle in between their nest and a foot source, they develop a path that loops around it. if you remove the obstacle, for some time they continue to follow the old looped path. However, some ants deviate and go around at random, exploring. eventually by chance one happens to find a quicker route. he gets a couple of his friends to follow him, by pheremone, and over time more and more take the quicker route, and they end up abandoning the old route It works this way with research, with most following the current trends, and some curious souls searching around for other ideas, be they contrarians, dreamers, or just convinced of some strange truth. But if we're right, signs tend to slowly begin to point their way, and we can shift the whole hulking edifice of science towards their point of view. The problem of llms is that while they may be able to find a shorter route, we can't follow them unless we understand the route. So the forces that slowly begin to change everyone's behavior are lost

CamperBob2 15d ago

The way you get out of a local maximum is to force yourself to try something radically new every so often, even if what you were doing before was working just fine. This is the 'stochastic' part of 'stochastic gradient descent,' and it's as important for human minds as it is for ANNs. These math wizards seem to be stuck in a rut of their own digging.