3 ms·

If mid-2026 LLMs were the last ones to ever exist, sure.

by meowface 8d ago

pmarreck 8d ago
and everything else is still speculation
- nomel 7d ago
  Sure, nobody knows the exact future. But you still plan for what's probably going to happen. I don't know if social security is going to be around when I retire, and all the experts are saying it's probably not going to be. So, I'm planning a retirement without. Similarly, all the experts are saying AI (<- that word is not "LLM") is probably going to improve. I personally think it's irrational to ignore experts.
  - pmarreck 7d ago
    I think that a person (me) who has worked directly with frontier AI for over a year now for many hours a day may have gained some expertise in what this technology is capable of and thus what is dangerous about it, but I wouldn't speculate too hard on what superintelligent AI would or could do because we have not yet seen such a thing nor is there even an established criteria as to what that would look like or how it would be valued and in what capacity (hint: humans are still the final evaluators of what is valuable) The simple fact that I have to write every control scheme in the book to keep the thing on the goal path (and not lapsing into laziness or dishonesty) is itself possibly a safety feature, if you look at it that way
    - nomel 5d ago
      > The simple fact that I have to write every control scheme ... Look three years ago for a reality check of the progress. The LLM you're using now isn't the LLM you'll be using in 5 years. It probably won't even really be the same architecture, just like those from 3 years ago are VERY different than the full systems we use today (harnesses and all). What you're doing was literal science fiction not long ago.
    - aswegs8 5d ago
      The fact that you work every day with frontier AI doesn't make you an expert on the future technological developments.
gandreani 8d ago
You're also assuming the next step towards AGI is an LLM. It's probably going to be another architecture entirely

If mid-2026 LLMs were the last ones to ever exist, sure.

pmarreck 8d ago

and everything else is still speculation

nomel 7d ago
Sure, nobody knows the exact future. But you still plan for what's probably going to happen. I don't know if social security is going to be around when I retire, and all the experts are saying it's probably not going to be. So, I'm planning a retirement without. Similarly, all the experts are saying AI (<- that word is not "LLM") is probably going to improve. I personally think it's irrational to ignore experts.
- pmarreck 7d ago
  I think that a person (me) who has worked directly with frontier AI for over a year now for many hours a day may have gained some expertise in what this technology is capable of and thus what is dangerous about it, but I wouldn't speculate too hard on what superintelligent AI would or could do because we have not yet seen such a thing nor is there even an established criteria as to what that would look like or how it would be valued and in what capacity (hint: humans are still the final evaluators of what is valuable) The simple fact that I have to write every control scheme in the book to keep the thing on the goal path (and not lapsing into laziness or dishonesty) is itself possibly a safety feature, if you look at it that way
  - nomel 5d ago
    > The simple fact that I have to write every control scheme ... Look three years ago for a reality check of the progress. The LLM you're using now isn't the LLM you'll be using in 5 years. It probably won't even really be the same architecture, just like those from 3 years ago are VERY different than the full systems we use today (harnesses and all). What you're doing was literal science fiction not long ago.
  - aswegs8 5d ago
    The fact that you work every day with frontier AI doesn't make you an expert on the future technological developments.

gandreani 8d ago

You're also assuming the next step towards AGI is an LLM. It's probably going to be another architecture entirely