4 ms·

“Next-token prediction” describes the output format, not the computation required to choose each token. During training, models develop internal representations

“Next-token prediction” describes the output format, not the computation required to choose each token. During training, models develop internal representations of concepts, constraints, possible futures, and algorithms. The PCB agent also writes circuit code, runs simulations, reads failures, and revises the design. It isn’t one-shot autocomplete. Astra and Fable are already hard to square with “mere autocomplete.” We may be (really) close to AGI, and token-by-token generation certainly doesn’t rule out subjective experience (I think we should at least treat that as an open question). Great videos: https://www.youtube.com/watch?v=D8GOeCFFby4 https://www.youtube.com/watch?v=D8GOeCFFby4 https://www.youtube.com/watch?v=Bj9BD2D3DzA https://www.youtube.com/watch?v=Bj9BD2D3DzA https://www.youtube.com/watch?v=l6DKRf-fAAM https://www.youtube.com/watch?v=l6DKRf-fAAM https://www.youtube.com/watch?v=GlYgs6v2YfU https://www.youtube.com/watch?v=GlYgs6v2YfU

kneyed 29d ago
right on! I like to say "token prediction is a task, not a limitation"
Zambyte 29d ago
How can you define "general" and "intelligence" in a way that has existed for years now?
- Zambyte 29d ago
  hasn't*
pineaux 29d ago
Yeah most of us are so fucked. With almost no way of protecting ourselves. No real amount of assets that will give enough power to save ourselves from the people in a position that can maximally leverage AI and lock others out. I see a future where these capabilities will be locked behind super high price pay walls. Why wouldn't they? How recoup investments if the price doesnt go up?
- brandnewideas 29d ago
  Shattering changes to societal structure are coming.
  - ultrarunner 29d ago
    It's like inflation for quality of life. Right now we're in the stimulus check phase, and people feel great about having been handed some cash. The next step is that cash has been spent, and what's remaining isn't worth anything.
    - gosub100 28d ago
      And just like welfare bucks, people will quickly grow accustomed to getting their "intelligence" handed to them from the machine, while their own intelligence atrophies. Breeding dependency
- deleted 28d ago
  [deleted]

kneyed 29d ago

right on! I like to say "token prediction is a task, not a limitation"

Zambyte 29d ago

How can you define "general" and "intelligence" in a way that has existed for years now?

Zambyte 29d ago
hasn't*

pineaux 29d ago

Yeah most of us are so fucked. With almost no way of protecting ourselves. No real amount of assets that will give enough power to save ourselves from the people in a position that can maximally leverage AI and lock others out. I see a future where these capabilities will be locked behind super high price pay walls. Why wouldn't they? How recoup investments if the price doesnt go up?

brandnewideas 29d ago
Shattering changes to societal structure are coming.
- ultrarunner 29d ago
  It's like inflation for quality of life. Right now we're in the stimulus check phase, and people feel great about having been handed some cash. The next step is that cash has been spent, and what's remaining isn't worth anything.
  - gosub100 28d ago
    And just like welfare bucks, people will quickly grow accustomed to getting their "intelligence" handed to them from the machine, while their own intelligence atrophies. Breeding dependency
deleted 28d ago
[deleted]