3 ms·

The challenge with estimating abilities, is that we don’t know what the models can achieve if we just burn enough money. The navier-stokes shows us what mathema

The challenge with estimating abilities, is that we don’t know what the models can achieve if we just burn enough money. The navier-stokes shows us what mathematical problem can be solved when $10m worth of compute is thrown at something. It also makes one wonder: What could AI solve if we managed to orchestrate billions worth of agents to take on a specific problem? IMO the very best case scenario / potential for these are likely better than we think, but right now hidden due to logistical and financial reasons. But if we assume that the model costs will continue to drop by a factor of 5-10 annually, there will always be a latency of a couple of years between what is completely out of reach, and what is financially viable. Basically: If you knew AI could be affordable enough in 3-5 years so that even the most underfunded researchers could use it to solve cancer, how much would you value it now?

dotdi 10d ago
The whole point of this post was that it's questionable what can be achieved without huge investments into oversight and steering, because navier-stokes was a topic with an unusual level of specification. The problem itself was a specification. Such situations are rare in real-world scenarios. AI agents are good at solving well-specified tasks, not at solving problems. They do well in fields where the cost/effort of specification is already part of the business.
- intrasight 10d ago
  > a topic with an unusual level of specification Solving cancer also has an unusual level of specification. Many real world problems have that characteristic.
  - lelanthran 10d ago
    > Solving cancer also has an unusual level of specification. Where did you read that? "Cancer" is not just a single disease, even though we layman use the term that way. Cancer is a family of diseases, each probably having their own specific solution, but even in each of these individual diseases, there is no specification at the level of any open maths problem.
    - intrasight 10d ago
      I know what cancer is. Many scientists believe that all human adults above the age of about 30 have "cancer". I really meant solving a specific cancer disease for a specific individual, which will require individualized medicine, which will require us to leverage AI to make it possible to do for the general population as opposed to doing it just for Lance Armstrong and the like. I know I'm an optimist, but I really think AI is gonna result in drastically improved healthcare for a much lower price.
  - hi_im_greg_h 10d ago
    This is so untrue its actually insane to me that anyone could believe this. The real world is so messy, and specifically cancer/biology is insanely messy and certainly not well specified. You should close chatgpt and read a book sometime.
  - goolz 10d ago
    Maybe a specific cancer but in general “solving cancer” is about as scattershot and unspecific as you could be.
andrepd 10d ago
> The navier-stokes shows us what mathematical problem can be solved when $10m worth of compute is thrown at something. That's the thing, it very much does NOT show us that. What happened was mathematicians at openAI learned of an imminent development on this problem, and the insight that it entailed, then they were able to prompt a system in the correct direction and spend 20 million dollars to write down the final steps. Which is rather precisely the point that the article is making! > If you knew AI could be affordable enough in 3-5 years so that even the most underfunded researchers could use it to solve cancer As the saying goes, if my grandmother had wheels she would have been a truck.
- famouswaffles 10d ago
  >What happened was mathematicians at openAI learned of an imminent development on this problem, and the insight that it entailed, then they were able to prompt a system in the correct direction and spend 20 million dollars to write down the final steps. That's not what happened.
  - andrepd 10d ago
    Actually it was. But thanks for elaborating.
    - famouswaffles 9d ago
      No it wasn't. They didn't know 'what direction' to take, and the solution they posted was not in fact the direction the authors took, so evidently you don't know what you're talking about. And they say LLMs hallucinate.
mitxela 10d ago
Wasn't Navier Stokes solved by ripping off a researcher's private chat log?
lelanthran 10d ago
> The navier-stokes shows us what mathematical problem can be solved when $10m worth of compute is thrown at something. > It also makes one wonder: What could AI solve if we managed to orchestrate billions worth of agents to take on a specific problem? We need to have robotics automation catchup first. The math and coding problems are problems in written-space only: you can set up feedback loops to test what worked and what didn't, then try to resolve the defects, maybe back up and try a different path, etc. What solved coding and maths problems weren't the damn models; open up a chat interface to a SOTA model and you'll see they are pretty limited in producing a solution without a feedback loop. Instead, it was the harness around the models: it let them explore a space and use feedback to control and direct that exploration. Until we can do it in meatspace, it's kinda pointless sinking a ton of money into large problems facing mankind... Like establishing a colony on mars (so the next rock to hit earth isn't an ELE). Or moving us to a post-scarcity utopia, ending the concept of money. Or designing and building better batteries for transport that uses only electricity (so that we stop using fossils as fuel). Or actually building mass-housing. Or mass-farming. Or both, potentially ending homelessness and starvation. Those are all worthwhile problems to solve, but where's the point of getting a solution on paper? There's no exploratory mechanism there, even for humans, to come up with a solution. So, all we are left with then is making knowledge workers obsolete: another ELE, but of a different, self-inflicted kind.
- suzzer99 10d ago
  You just invented Skynet.

dotdi 10d ago

The whole point of this post was that it's questionable what can be achieved without huge investments into oversight and steering, because navier-stokes was a topic with an unusual level of specification. The problem itself was a specification. Such situations are rare in real-world scenarios. AI agents are good at solving well-specified tasks, not at solving problems. They do well in fields where the cost/effort of specification is already part of the business.

intrasight 10d ago
> a topic with an unusual level of specification Solving cancer also has an unusual level of specification. Many real world problems have that characteristic.
- lelanthran 10d ago
  > Solving cancer also has an unusual level of specification. Where did you read that? "Cancer" is not just a single disease, even though we layman use the term that way. Cancer is a family of diseases, each probably having their own specific solution, but even in each of these individual diseases, there is no specification at the level of any open maths problem.
  - intrasight 10d ago
    I know what cancer is. Many scientists believe that all human adults above the age of about 30 have "cancer". I really meant solving a specific cancer disease for a specific individual, which will require individualized medicine, which will require us to leverage AI to make it possible to do for the general population as opposed to doing it just for Lance Armstrong and the like. I know I'm an optimist, but I really think AI is gonna result in drastically improved healthcare for a much lower price.
- hi_im_greg_h 10d ago
  This is so untrue its actually insane to me that anyone could believe this. The real world is so messy, and specifically cancer/biology is insanely messy and certainly not well specified. You should close chatgpt and read a book sometime.
- goolz 10d ago
  Maybe a specific cancer but in general “solving cancer” is about as scattershot and unspecific as you could be.

andrepd 10d ago

> The navier-stokes shows us what mathematical problem can be solved when $10m worth of compute is thrown at something. That's the thing, it very much does NOT show us that. What happened was mathematicians at openAI learned of an imminent development on this problem, and the insight that it entailed, then they were able to prompt a system in the correct direction and spend 20 million dollars to write down the final steps. Which is rather precisely the point that the article is making! > If you knew AI could be affordable enough in 3-5 years so that even the most underfunded researchers could use it to solve cancer As the saying goes, if my grandmother had wheels she would have been a truck.

famouswaffles 10d ago
>What happened was mathematicians at openAI learned of an imminent development on this problem, and the insight that it entailed, then they were able to prompt a system in the correct direction and spend 20 million dollars to write down the final steps. That's not what happened.
- andrepd 10d ago
  Actually it was. But thanks for elaborating.
  - famouswaffles 9d ago
    No it wasn't. They didn't know 'what direction' to take, and the solution they posted was not in fact the direction the authors took, so evidently you don't know what you're talking about. And they say LLMs hallucinate.

mitxela 10d ago

Wasn't Navier Stokes solved by ripping off a researcher's private chat log?

lelanthran 10d ago

> The navier-stokes shows us what mathematical problem can be solved when $10m worth of compute is thrown at something. > It also makes one wonder: What could AI solve if we managed to orchestrate billions worth of agents to take on a specific problem? We need to have robotics automation catchup first. The math and coding problems are problems in written-space only: you can set up feedback loops to test what worked and what didn't, then try to resolve the defects, maybe back up and try a different path, etc. What solved coding and maths problems weren't the damn models; open up a chat interface to a SOTA model and you'll see they are pretty limited in producing a solution without a feedback loop. Instead, it was the harness around the models: it let them explore a space and use feedback to control and direct that exploration. Until we can do it in meatspace, it's kinda pointless sinking a ton of money into large problems facing mankind... Like establishing a colony on mars (so the next rock to hit earth isn't an ELE). Or moving us to a post-scarcity utopia, ending the concept of money. Or designing and building better batteries for transport that uses only electricity (so that we stop using fossils as fuel). Or actually building mass-housing. Or mass-farming. Or both, potentially ending homelessness and starvation. Those are all worthwhile problems to solve, but where's the point of getting a solution on paper? There's no exploratory mechanism there, even for humans, to come up with a solution. So, all we are left with then is making knowledge workers obsolete: another ELE, but of a different, self-inflicted kind.

suzzer99 10d ago
You just invented Skynet.