3 ms·

The problem is that eventually, there will be no humans who can follow the results AI will give. That’s the endgame for this tech: to produce knowledge at speed

The problem is that eventually, there will be no humans who can follow the results AI will give. That’s the endgame for this tech: to produce knowledge at speeds and quality beyond what we can

uargos 16d ago
Which asks the question of what knowledge is. Can knowledge be super human ? Or is knowledge a human matter ? If so (like i believe), then what those ai labs are doing is far from the end of the story. Because the goal of science is not to produce a certificate of something, but more to produce an explanation that can fit in a human brain, that can be reasoned on, and that can be retargeted. In this sense, producing a million lines proof is not really producing knowledge, even less so doing science.
- michaeljx 16d ago
  Knowing something is proven true is knowledge, even if the mechanics of the proof are not understood.
  - rented_mule 16d ago
    If the mechanics of the proof are not understood, how do we know it's been proven? Math proofs are not a "trust me, bro" kind of thing.
  - trolleski 16d ago
    In maths, you go through proofs every step of the way. Same in physics, you start by performing even the most basic experiments, and build your way up.
  - uargos 16d ago
    It's not knowledge, it's belief.
- curt15 16d ago
  Suppose there is an oracle that can tell you whether a particular result in maths or physics is true. Would research still have any value?
smitty1e 16d ago
Yeah, but then you can use another round of AI to break these things down to crayon level, no? Or is it just all PFM? (Pure, Fanciful Magic)
menaerus 16d ago
In certain, more complex, software engineering domains this almost became true as of today.
rsfern 16d ago
That’s the prevailing narrative, but I think this controversy calls it into question to some extent. If the OpenAI result wouldn’t have been possible without experts seeding the training data with feedback on promising solution routes, there’s less reason to believe this, IMO. More information and transparency is needed
- bot403 16d ago
  That's the pickle isn't it? AI solved it with human help. But it's standing on their shoulders. Neither AI nor the humans got it alone. Humanity is better having solved this issue. But which humans were credited and benefited is the issue.
  - rsfern 16d ago
    I agree (and so does Buckmaster based on his written statement) that we are better having solved this. But I disagree that which humans were credited is the heart of the issue in this particular controversy. The question is what do you need to bring to the table for a result like this. A pre-release frontier model trained on the open literature and $15 million of inference? Or all that plus a year of the experts finding the path to the solution for the model to run with? I think it makes a huge difference in terms of what we think the future of mathematical research will be like, and whether we should still encourage students to go into this field, which was the original topic of this thread
  - bot404 16d ago
    [dead]

The problem is that eventually, there will be no humans who can follow the results AI will give. That’s the endgame for this tech: to produce knowledge at speeds and quality beyond what we can

uargos 16d ago

Which asks the question of what knowledge is. Can knowledge be super human ? Or is knowledge a human matter ? If so (like i believe), then what those ai labs are doing is far from the end of the story. Because the goal of science is not to produce a certificate of something, but more to produce an explanation that can fit in a human brain, that can be reasoned on, and that can be retargeted. In this sense, producing a million lines proof is not really producing knowledge, even less so doing science.

michaeljx 16d ago
Knowing something is proven true is knowledge, even if the mechanics of the proof are not understood.
- rented_mule 16d ago
  If the mechanics of the proof are not understood, how do we know it's been proven? Math proofs are not a "trust me, bro" kind of thing.
- trolleski 16d ago
  In maths, you go through proofs every step of the way. Same in physics, you start by performing even the most basic experiments, and build your way up.
- uargos 16d ago
  It's not knowledge, it's belief.
curt15 16d ago
Suppose there is an oracle that can tell you whether a particular result in maths or physics is true. Would research still have any value?

smitty1e 16d ago

Yeah, but then you can use another round of AI to break these things down to crayon level, no? Or is it just all PFM? (Pure, Fanciful Magic)

menaerus 16d ago

In certain, more complex, software engineering domains this almost became true as of today.

rsfern 16d ago

That’s the prevailing narrative, but I think this controversy calls it into question to some extent. If the OpenAI result wouldn’t have been possible without experts seeding the training data with feedback on promising solution routes, there’s less reason to believe this, IMO. More information and transparency is needed

bot403 16d ago
That's the pickle isn't it? AI solved it with human help. But it's standing on their shoulders. Neither AI nor the humans got it alone. Humanity is better having solved this issue. But which humans were credited and benefited is the issue.
- rsfern 16d ago
  I agree (and so does Buckmaster based on his written statement) that we are better having solved this. But I disagree that which humans were credited is the heart of the issue in this particular controversy. The question is what do you need to bring to the table for a result like this. A pre-release frontier model trained on the open literature and $15 million of inference? Or all that plus a year of the experts finding the path to the solution for the model to run with? I think it makes a huge difference in terms of what we think the future of mathematical research will be like, and whether we should still encourage students to go into this field, which was the original topic of this thread
- bot404 16d ago
  [dead]