3 ms·

Something that actually could predict the next token 100% correctly would be omniscient. So I hardly see why this is inherently crazy. At most I think it might

Something that actually could predict the next token 100% correctly would be omniscient. So I hardly see why this is inherently crazy. At most I think it might not be scalable.

edude03 2y ago
What does it mean to predict the next token correctly though? Arguably (non instruction tuned) models already regurgitate their training data such that it'd complete "Mary had a" with "little lamb" 100% of the time. On the other hand if you mean, give you the correct answer to your question 100% of the time, then I agree, though then what about things that are only in your mind (guess the number I'm thinking type problems)?
- card_zero 2y ago
  This highlights something that's wrong about arguments for AI. I say: it's not human-like intelligence, it's just predicting the next token probabilistically. Some AI advocate says: humans are just predicting the next token probabilistically, fight me. The problem here is that "predicting the next token probabilistically" is a way of framing any kind of cleverness, up to and including magical, impossible omniscience. That doesn't mean it's the way every kind of cleverness is actually done, or could realistically be done. And it has to be the correct next token, where all the details of what's actually required are buried in that term "correct", and sometimes it literally means the same as "likely", and other times that just produces a reasonable, excusable, intelligence-esque effort.
  - dylan604 2y ago
    > Some AI advocate says: humans are just predicting the next token probabilistically, fight me. We've all had conversations with humans that are always jumping to complete your sentence assuming they know what your about to say and don't quite guess correctly. So AI evangelists are saying it's no worse than humans as their proof. I kind of like their logic. They never claimed to have built HAL /s
    - card_zero 2y ago
      No worse than a human on autopilot.
  - usaar333 2y ago
    https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-general-intelligence/ https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-g... This essay has aged extremely well.
- cruffle_duffle 2y ago
  But now you are entering into philosophy. What does a “correct answer” even mean for a question like “is it safe to lick your fingers after using a soldering iron with leaded solder?”. I would assert that there is no “correct answer” to a question like that. Is it safe? Probably. But it depends, right? How did you handle the solder? How often are you using the solder? Were you wearing gloves? Did you wash your hands before licking your fingers? What is your age? Why are you asking the question? Did you already lick your fingers and need to know if you should see a doctor? Is it hypothetical? There is no “correct answer” to that question. Some answers are better than others, yes, but you cannot have a “correct answer”. And I did assert we are entering into philosophy and what it means to know something as well as what truth even means.
  - _blk 2y ago
    Great break-down. Yes, the older you are, the safer it is. Speaking of Microsoft cooperation: I can totally see a whole series of windows 95 style popup dialogs asking you all those questions one by one in the next product iteration.
- usaar333 2y ago
  > What does it mean to predict the next token correctly though? Arguably (non instruction tuned) models already regurgitate their training data such that it'd complete "Mary had a" with "little lamb" 100% of the time. The unseen test data. Obviously omniscience is physically impossible. The point though is that the better and better next token prediction is, the more intelligent the system must be.
deleted 2y ago
[deleted]
sksxihve 2y ago
It's not possible for the same reason the halting problem is undecidable.
Vegenoid 2y ago
Start by trying to define what “100% correct” means in the context of predicting the next token, and the flaws with this line of thinking should reveal themselves.
kbrkbr 2y ago
That does not seem to be true. Either the next tokens can include "this question can't be answered", "I don't know" and the likes, in which case there is no omniscience. Or the next tokens must contain answers that do not go on the meta level, but only pick one of the potential direct answers to a question. Then the halting problem will prevent finite time omniscience (which is, from the perspective of finite beings all omniscience).

Something that actually could predict the next token 100% correctly would be omniscient. So I hardly see why this is inherently crazy. At most I think it might not be scalable.

edude03 2y ago

What does it mean to predict the next token correctly though? Arguably (non instruction tuned) models already regurgitate their training data such that it'd complete "Mary had a" with "little lamb" 100% of the time. On the other hand if you mean, give you the correct answer to your question 100% of the time, then I agree, though then what about things that are only in your mind (guess the number I'm thinking type problems)?

card_zero 2y ago
This highlights something that's wrong about arguments for AI. I say: it's not human-like intelligence, it's just predicting the next token probabilistically. Some AI advocate says: humans are just predicting the next token probabilistically, fight me. The problem here is that "predicting the next token probabilistically" is a way of framing any kind of cleverness, up to and including magical, impossible omniscience. That doesn't mean it's the way every kind of cleverness is actually done, or could realistically be done. And it has to be the correct next token, where all the details of what's actually required are buried in that term "correct", and sometimes it literally means the same as "likely", and other times that just produces a reasonable, excusable, intelligence-esque effort.
- dylan604 2y ago
  > Some AI advocate says: humans are just predicting the next token probabilistically, fight me. We've all had conversations with humans that are always jumping to complete your sentence assuming they know what your about to say and don't quite guess correctly. So AI evangelists are saying it's no worse than humans as their proof. I kind of like their logic. They never claimed to have built HAL /s
  - card_zero 2y ago
    No worse than a human on autopilot.
- usaar333 2y ago
  https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-general-intelligence/ https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-g... This essay has aged extremely well.
cruffle_duffle 2y ago
But now you are entering into philosophy. What does a “correct answer” even mean for a question like “is it safe to lick your fingers after using a soldering iron with leaded solder?”. I would assert that there is no “correct answer” to a question like that. Is it safe? Probably. But it depends, right? How did you handle the solder? How often are you using the solder? Were you wearing gloves? Did you wash your hands before licking your fingers? What is your age? Why are you asking the question? Did you already lick your fingers and need to know if you should see a doctor? Is it hypothetical? There is no “correct answer” to that question. Some answers are better than others, yes, but you cannot have a “correct answer”. And I did assert we are entering into philosophy and what it means to know something as well as what truth even means.
- _blk 2y ago
  Great break-down. Yes, the older you are, the safer it is. Speaking of Microsoft cooperation: I can totally see a whole series of windows 95 style popup dialogs asking you all those questions one by one in the next product iteration.
usaar333 2y ago
> What does it mean to predict the next token correctly though? Arguably (non instruction tuned) models already regurgitate their training data such that it'd complete "Mary had a" with "little lamb" 100% of the time. The unseen test data. Obviously omniscience is physically impossible. The point though is that the better and better next token prediction is, the more intelligent the system must be.

deleted 2y ago

[deleted]

sksxihve 2y ago

It's not possible for the same reason the halting problem is undecidable.

Vegenoid 2y ago

Start by trying to define what “100% correct” means in the context of predicting the next token, and the flaws with this line of thinking should reveal themselves.

kbrkbr 2y ago

That does not seem to be true. Either the next tokens can include "this question can't be answered", "I don't know" and the likes, in which case there is no omniscience. Or the next tokens must contain answers that do not go on the meta level, but only pick one of the potential direct answers to a question. Then the halting problem will prevent finite time omniscience (which is, from the perspective of finite beings all omniscience).