5 ms·

Terrence Tao: AI Is Teaching Us Something Uncomfortable About Our Own Minds [video]

17 points by thelastgallon 1mo ago

aaron695 1mo ago
[dead]
qarl 1mo ago
I despise the "next token predictor" framing. In order to predict the next token in this line "The solution to your problem is..." the entity needs to be able to solve your problem. Which is well beyond what people think of when they think of a token predicting statistical sequence. The statistical sequence description is ridiculously incomplete.
- anon-3988 1mo ago
  Next token predictor is as useful as saying that humans are just atoms and neurones. It is technically true but meaningless.
  - qarl 1mo ago
    An LLM agent is literally Turing complete. There is nothing it cannot compute. Anytime you hear someone saying "it can't do this, or that..." they are factually incorrect. The denial around these issues is beyond creepy: AI has clearly hit some tender nerve in the human mind.
    - TophWells 1mo ago
      Source? Turing completeness is a very specific mathematical definition, it would surprise me if a neural network met it. Real-world computers are only Turing complete if you assume that they have unlimited memory space -- which is a reasonable approximation, but technically not true.
      - qarl 1mo ago
        Yes. If we allow a file with unbounded storage (like your exception for physical computers) then an agent can easily perform the necessary state transitions and tape reads/writes.
- EliRivers 1mo ago
  the entity needs to be able to solve your problem. People have fixed software bugs with zero knowledge of what the code is doing and no knowledge of the programming language at hand, purely by statistical pattern matching. Solving the problem with no understanding of it beyond the idea that whatever it was, the right answer was probably something that kind of matched the kind of patterns found in things they'd seen before. So what does it mean, "to be able to solve your problem"?
  - qarl 1mo ago
    > People have fixed software bugs with zero knowledge of what the code is doing and no knowledge of the programming language at hand, purely by statistical pattern matching. Is that even true? Beyond an extremely simple example?
    - EliRivers 27d ago
      Yes, it is. Beyond simple examples, I've fixed segfaults by identifying the assembly where it happens, tracing that back to the lump of source where it happens, guessed which piece is accessing memory and deducing from surrounding code how it was meant to be protected against. In that case, the pattern matching was against every segfault I've ever fixed, which is a lot, and traipsing through assembly, which is also a lot. Lots of experience in tracing back to source code. So did that. Matching patterns. Didn't understand the language, don't know what it's trying to do, but I know what memory access is (because I've seen the pattern "access memory" a lot in my life) and this piece of code must be matching that pattern; whatever this code is, it must be touching memory. Now scroll up and down to look for similar code in other places, that I also cannot read but I can see when things look the same; match the pattern of pieces that work with this piece that doesn't. Oh, this looks the same but it has this extra piece; part of the pattern matches, part does not. Matching patterns. What if I copy it, paste into the bad place, apply a single change to match the symbols I see in the bad place, and see what happens... fixed. It's not something I do every day and it's not nearly as good as actually understanding it, but the problem was solved and I did not know what the code was trying to do or how to read it. All the above aside, surely it must be possible, because we have machines that do exactly that. They're made of statistics and patterns and they're doing it. Right in front of us. There is nothing else in there and they're doing it right now.
ETH_start 1mo ago
Going off on a tangent here: for over a decade, ever since I got an inkling of the power of neural networks, I've argued that privacy is the key to maintaining a broad diffusion of wealth in the modern electronically networked world. What distinguishes human intelligence from LLM intelligence is largely information that the latter hasn't mopped up. So the information that the former is uniquely privy to is what gives it economic value.

17 points by thelastgallon 1mo ago

[dead]

qarl 1mo ago

I despise the "next token predictor" framing. In order to predict the next token in this line "The solution to your problem is..." the entity needs to be able to solve your problem. Which is well beyond what people think of when they think of a token predicting statistical sequence. The statistical sequence description is ridiculously incomplete.

anon-3988 1mo ago
Next token predictor is as useful as saying that humans are just atoms and neurones. It is technically true but meaningless.
- qarl 1mo ago
  An LLM agent is literally Turing complete. There is nothing it cannot compute. Anytime you hear someone saying "it can't do this, or that..." they are factually incorrect. The denial around these issues is beyond creepy: AI has clearly hit some tender nerve in the human mind.
  - TophWells 1mo ago
    Source? Turing completeness is a very specific mathematical definition, it would surprise me if a neural network met it. Real-world computers are only Turing complete if you assume that they have unlimited memory space -- which is a reasonable approximation, but technically not true.
    - qarl 1mo ago
      Yes. If we allow a file with unbounded storage (like your exception for physical computers) then an agent can easily perform the necessary state transitions and tape reads/writes.
EliRivers 1mo ago
the entity needs to be able to solve your problem. People have fixed software bugs with zero knowledge of what the code is doing and no knowledge of the programming language at hand, purely by statistical pattern matching. Solving the problem with no understanding of it beyond the idea that whatever it was, the right answer was probably something that kind of matched the kind of patterns found in things they'd seen before. So what does it mean, "to be able to solve your problem"?
- qarl 1mo ago
  > People have fixed software bugs with zero knowledge of what the code is doing and no knowledge of the programming language at hand, purely by statistical pattern matching. Is that even true? Beyond an extremely simple example?
  - EliRivers 27d ago
    Yes, it is. Beyond simple examples, I've fixed segfaults by identifying the assembly where it happens, tracing that back to the lump of source where it happens, guessed which piece is accessing memory and deducing from surrounding code how it was meant to be protected against. In that case, the pattern matching was against every segfault I've ever fixed, which is a lot, and traipsing through assembly, which is also a lot. Lots of experience in tracing back to source code. So did that. Matching patterns. Didn't understand the language, don't know what it's trying to do, but I know what memory access is (because I've seen the pattern "access memory" a lot in my life) and this piece of code must be matching that pattern; whatever this code is, it must be touching memory. Now scroll up and down to look for similar code in other places, that I also cannot read but I can see when things look the same; match the pattern of pieces that work with this piece that doesn't. Oh, this looks the same but it has this extra piece; part of the pattern matches, part does not. Matching patterns. What if I copy it, paste into the bad place, apply a single change to match the symbols I see in the bad place, and see what happens... fixed. It's not something I do every day and it's not nearly as good as actually understanding it, but the problem was solved and I did not know what the code was trying to do or how to read it. All the above aside, surely it must be possible, because we have machines that do exactly that. They're made of statistics and patterns and they're doing it. Right in front of us. There is nothing else in there and they're doing it right now.

ETH_start 1mo ago

Going off on a tangent here: for over a decade, ever since I got an inkling of the power of neural networks, I've argued that privacy is the key to maintaining a broad diffusion of wealth in the modern electronically networked world. What distinguishes human intelligence from LLM intelligence is largely information that the latter hasn't mopped up. So the information that the former is uniquely privy to is what gives it economic value.