3 ms·

I think this might be a red herring. All it takes is someone to get an inkling that someone is working on a new approach and seeing some success for OpenAI to f

I think this might be a red herring. All it takes is someone to get an inkling that someone is working on a new approach and seeing some success for OpenAI to fire the AI cannon at the problem. The community seems fairly small (from this outsider's point of view). The idea that the data made it into the training set and that's how the bot figured it out is definitely possible, but I would want to rule out the simpler more direct explanation first. The fact that this academic sniping can now be done at scale does change the formula though and shouldn't be ignored. The pressure to move math work into secrecy because at the slightest signal OpenAI and Anthropic will start burning tokens for headlines, is bad for math and its bad for everyone.

zeven7 26d ago
Terence Tao said the same[1] > In fact, it is now the identification of a promising problem which is the scarce and precious resource. We have now seen that even the rumor of someone working on a problem can trigger a massive amount of AI-powered effort to flatten it before the original research project has time to reach its full potential. The incentives may now be pointing in the direction of no longer sharing any promising research directions with the broader community, which would reverse centuries of traditions of open science and do serious long-term damage to the future of the field. [1] https://mathstodon.xyz/@tao/117237322160500501 https://mathstodon.xyz/@tao/117237322160500501
- zactato 26d ago
  Could you just start engineering "leaks" of new proofs so that Anthropic or OpenAI just start burning $10million in compute
- tgma 26d ago
  > it is now the identification of a promising problem which is the scarce and precious resource This is by no means new. Perhaps it is even more extreme now. Literally my first 1:1 with my PhD adviser back then, he told me that the most important thing about a researcher is the quality of the problems he picks.
  - tomcorrigan 26d ago
    Sure; if you define the quality of a problem by reference to your ability to solve it. In the real world the quality of these esoteric problems is typically gauged by the difficulty of solving them.
    - tgma 26d ago
      > In the real world the quality of these esoteric problems is typically gauged by the difficulty of solving them. I don't believe that's strictly true. Way oversimplified projection on one axis.
- conorbergin 26d ago
  This reminds of Magnus Carlsen saying that if he were to cheat, all he would need would be a signal to spend more time on the current move.
- geraneum 26d ago
  Shouldn’t this very capable model they’ve developed be able to identify promising problems? That’s what I’d expect from how the model is being presented and advertised.
  - willsmith72 26d ago
    i've seen no sign of that
  - sigmoid10 26d ago
    That's more or less what they did according to their announcement. They fired it at a whole bunch of high end math problems and merely concentrated all efforts on one after it made some promising progress.
    - geraneum 26d ago
      It seems like that’s the opposite of what happened. They started attacking the problem when they got a wind of a possible solution from certain individuals.
      - sigmoid10 24d ago
        But they didn't know which problem had a possible solution. So they fired it at a huge amount of problems and then merely focused on the one that turned out to be promising. The model found the promising path itself, they just re-allocated the available resources once it became apparent.
eaurouge 26d ago
That's pretty much it. This is a bit like (but worse imo) running a vc firm, listening (formally or informally) to idea pitches, then spinning out and funding competitors with millions of dollars, to outcompete the originators of those ideas. Yes, the idea is not secret in this case, but the unfair advantage is massive and there's only one (a couple at most) positive outcome.

zeven7 26d ago

Terence Tao said the same[1] > In fact, it is now the identification of a promising problem which is the scarce and precious resource. We have now seen that even the rumor of someone working on a problem can trigger a massive amount of AI-powered effort to flatten it before the original research project has time to reach its full potential. The incentives may now be pointing in the direction of no longer sharing any promising research directions with the broader community, which would reverse centuries of traditions of open science and do serious long-term damage to the future of the field. [1] https://mathstodon.xyz/@tao/117237322160500501 https://mathstodon.xyz/@tao/117237322160500501

zactato 26d ago
Could you just start engineering "leaks" of new proofs so that Anthropic or OpenAI just start burning $10million in compute
tgma 26d ago
> it is now the identification of a promising problem which is the scarce and precious resource This is by no means new. Perhaps it is even more extreme now. Literally my first 1:1 with my PhD adviser back then, he told me that the most important thing about a researcher is the quality of the problems he picks.
- tomcorrigan 26d ago
  Sure; if you define the quality of a problem by reference to your ability to solve it. In the real world the quality of these esoteric problems is typically gauged by the difficulty of solving them.
  - tgma 26d ago
    > In the real world the quality of these esoteric problems is typically gauged by the difficulty of solving them. I don't believe that's strictly true. Way oversimplified projection on one axis.
conorbergin 26d ago
This reminds of Magnus Carlsen saying that if he were to cheat, all he would need would be a signal to spend more time on the current move.
geraneum 26d ago
Shouldn’t this very capable model they’ve developed be able to identify promising problems? That’s what I’d expect from how the model is being presented and advertised.
- willsmith72 26d ago
  i've seen no sign of that
- sigmoid10 26d ago
  That's more or less what they did according to their announcement. They fired it at a whole bunch of high end math problems and merely concentrated all efforts on one after it made some promising progress.
  - geraneum 26d ago
    It seems like that’s the opposite of what happened. They started attacking the problem when they got a wind of a possible solution from certain individuals.
    - sigmoid10 24d ago
      But they didn't know which problem had a possible solution. So they fired it at a huge amount of problems and then merely focused on the one that turned out to be promising. The model found the promising path itself, they just re-allocated the available resources once it became apparent.

eaurouge 26d ago

That's pretty much it. This is a bit like (but worse imo) running a vc firm, listening (formally or informally) to idea pitches, then spinning out and funding competitors with millions of dollars, to outcompete the originators of those ideas. Yes, the idea is not secret in this case, but the unfair advantage is massive and there's only one (a couple at most) positive outcome.