2 ms·

> Calling it a token predictor is just like saying a computer is a bit mover. Calling it a token-predictor isn't reductionism. It's designed, implemented and t

> Calling it a token predictor is just like saying a computer is a bit mover. Calling it a token-predictor isn't reductionism. It's designed, implemented and trained for token prediction. Training means that the weights are adjusted in the network until it accurately predicts tokens. Predicting a token is something along the lines of removing a word from a sentence and getting it to predict it back: "The quick brown fox jumped over the lazy ____". Correct prediction is "dogs". So actually it is like calling a grass-cutting machine "lawn mower". > I mean we can’t even characterize definitively what consciousness is at the language level. But, oh, just believe the LLM when it produces a sentence referring to itself, claiming it is conscious.

ninetyninenine 1y ago
>Calling it a token-predictor isn't reductionism. It's designed, implemented and trained for token prediction. Training means that the weights are adjusted in the network until it accurately predicts tokens. Predicting a token is something along the lines of removing a word from a sentence and getting it to predict it back: "The quick brown fox jumped over the lazy ____". Correct prediction is "dogs". It absolutely is reductionism. Ask any expert who knows how these things work and they will say the same: https://youtu.be/qrvK_KuIeJk?t=497 https://youtu.be/qrvK_KuIeJk?t=497 Above we have Geoffrey Hinton, the godfather of the current wave of AI saying your statements are absolutely crazy. It's nuts that I don't actually have to offer any proof to convince you. Proof won't convince you. I just have to show you someone smarter than you with a better reputation saying the exact opposite and that is what flips you. Human psychology readily can attack logic and rationality. You can scaffold any amount of twisted logic and irrelevant analogies to get around any bulwark to support your own point. Human psychology fails when attacking another person who has a higher rank. Going against someone of higher rank this causes you to think twice and rethink your own position. In debates, logic is ineffective, bringing opposing statements with (while offering ZERO concrete evidence) from experts with a higher rank is the actual way to convince you. >But, oh, just believe the LLM when it produces a sentence referring to itself, claiming it is conscious. This is an hallucination. Showing that you're not much different from an LLM. I NEVER stated this. I said it's possible. But I said we cannot make a definitive statement either way. We cannot say it isn't conscious we cannot say it is. First we don't understand the LLM and second WE don't even have an exact definition of consciousness. So to say it's not conscious is JUST as ludicrous as saying it is. Understand?
- bigyabai 1y ago
  > the godfather of the current wave of AI Here we go again... Geoffrey Hinton is popular because of ML. You know why he doesn't understand the modern wave of LLMs? Because the transformer architecture isn't his invention. He had almost nothing to do with it. It's like saying that nobody understands how the nuclear bomb works simply because Einstein won't explain it to you in detail. There is a science here, and denying that it can be interpreted or understood is simply hysterical. We can perfectly well understand a Markov chain, why are LLMs any different? The "current wave of AI" is an autoregressive feed-forward algorithm inferring tokenized data using complex pretrained weights. In simple terms, you could describe this as a token predictor without glossing over any details that are fundamentally important. To call it "reductionist" you would have to explain what is missing from the definition, not simply that you distrust people who understand it.
  - ninetyninenine 1y ago
    [flagged]
  - DonHopkins 1y ago
    Unless I'm mistaken, the transformer architecture isn't your invention either, yet you claim to understand it better than Geoffrey Hinton. Did you invent the transformer architecture yourself, and I am mistaken to guess that you probably understand it less than Geoffrey Hinton does? I'll venture a wild guess that you probably haven't been working in the field as long as he has, and you don't personally know as many people in the field as he does, huh? Or is my presumptuous chain of logic just a random thoughtless unconscious probabilistic stream of tokens that has nothing to do with reality, and I don't realize I'm speaking to Ashish Vaswani himself? Then why don't you use your real name as you HN handle, are you really that modest, Ashish? And don't you have better things to do than posting to Hacker News? If you really wanted to win an argument from authority, you could at least post under your real name, Mr. Vaswani.
DonHopkins 1y ago
>"The quick brown fox jumped over the lazy ____". Correct prediction is "dogs". I would have predicted a single lazy "dog". Does that mean I'm more conscious than you? ;)
- kazinator 1y ago
  You don't get to pick what is "correct"; that's dictated by the training data. (Which cuold contain multiple versions that have either choice).
  - DonHopkins 1y ago
    Neither do you, yet you literally just brashly and overconfidently hallucinated exactly like a poorly trained token predicting LLM that the: >Correct prediction is "dogs". Maybe I just trained on Wikipedia, or the February 9, 1885 edition of The Boston Journal "Current Notes" article, or Linda Bronson's 1888 book "Illustrative Shorthand", for what the statistically and historically "correct" completion is. https://en.wikipedia.org/wiki/The_quick_brown_fox_jumps_over_the_lazy_dog https://en.wikipedia.org/wiki/The_quick_brown_fox_jumps_over... [..._the_lazy_dog] Why would you insert redundant letters into a lovely well-known succinct pangram, when the whole point is to make the SHORTEST sentence with all letters of the alphabet? This entire post could also technically be considered a terribly redundant pangram if you wanted to throw in a lot of unnecessary letters, slashes, colons, underscores, double quotes, commas, periods, question marks, brackets, equal signs, digits, spaces, and newlines. As could Moby Dick, War and Peace, or the Unabomber Manifesto. >So actually it is like calling a grass-cutting machine "lawn mower". Is it anything like calling Larry Ellison a "lawn mower", which is much more "correct" than anthropomorphising him? https://news.ycombinator.com/item?id=15886728 https://news.ycombinator.com/item?id=15886728 As ninetyninenine pointed out, we don't understand what's going on with you either: >Someone can claim that the responses are an illusion and they have no meaning but that exact argument applies to humans and we can’t say anything either way. We don’t understand what’s going on.
    - kazinator 1y ago
      What I meant was that the correct prediction is "dogs" under the assumption that that is what has occurred in the arbitrarily selected training data. While we are training the thing, we want to get it to predict whatever is in the data. > Why would you insert redundant letters into a lovely well-known succinct pangram The 's' is not redundant, because my version of the pangram has "fox jumped" not "fox jumps". I picked up this version as a kid and have always used that one when testing new keyboards. > the whole point is to make the SHORTEST sentence with all letters of the alphabet If that is the whole point, the fox-dog sentence must be abandoned for a shorter pangram. The requirement those shorter pangrams tend not to satisfy is being easy to remember. People don't care about the sentence being short, which is why variants starting with "The ..." instead of "A ..." soon appeared after the invention of the original, which itself could have dropped the "A".

ninetyninenine 1y ago

>Calling it a token-predictor isn't reductionism. It's designed, implemented and trained for token prediction. Training means that the weights are adjusted in the network until it accurately predicts tokens. Predicting a token is something along the lines of removing a word from a sentence and getting it to predict it back: "The quick brown fox jumped over the lazy ____". Correct prediction is "dogs". It absolutely is reductionism. Ask any expert who knows how these things work and they will say the same: https://youtu.be/qrvK_KuIeJk?t=497 https://youtu.be/qrvK_KuIeJk?t=497 Above we have Geoffrey Hinton, the godfather of the current wave of AI saying your statements are absolutely crazy. It's nuts that I don't actually have to offer any proof to convince you. Proof won't convince you. I just have to show you someone smarter than you with a better reputation saying the exact opposite and that is what flips you. Human psychology readily can attack logic and rationality. You can scaffold any amount of twisted logic and irrelevant analogies to get around any bulwark to support your own point. Human psychology fails when attacking another person who has a higher rank. Going against someone of higher rank this causes you to think twice and rethink your own position. In debates, logic is ineffective, bringing opposing statements with (while offering ZERO concrete evidence) from experts with a higher rank is the actual way to convince you. >But, oh, just believe the LLM when it produces a sentence referring to itself, claiming it is conscious. This is an hallucination. Showing that you're not much different from an LLM. I NEVER stated this. I said it's possible. But I said we cannot make a definitive statement either way. We cannot say it isn't conscious we cannot say it is. First we don't understand the LLM and second WE don't even have an exact definition of consciousness. So to say it's not conscious is JUST as ludicrous as saying it is. Understand?

bigyabai 1y ago
> the godfather of the current wave of AI Here we go again... Geoffrey Hinton is popular because of ML. You know why he doesn't understand the modern wave of LLMs? Because the transformer architecture isn't his invention. He had almost nothing to do with it. It's like saying that nobody understands how the nuclear bomb works simply because Einstein won't explain it to you in detail. There is a science here, and denying that it can be interpreted or understood is simply hysterical. We can perfectly well understand a Markov chain, why are LLMs any different? The "current wave of AI" is an autoregressive feed-forward algorithm inferring tokenized data using complex pretrained weights. In simple terms, you could describe this as a token predictor without glossing over any details that are fundamentally important. To call it "reductionist" you would have to explain what is missing from the definition, not simply that you distrust people who understand it.
- ninetyninenine 1y ago
  [flagged]
- DonHopkins 1y ago
  Unless I'm mistaken, the transformer architecture isn't your invention either, yet you claim to understand it better than Geoffrey Hinton. Did you invent the transformer architecture yourself, and I am mistaken to guess that you probably understand it less than Geoffrey Hinton does? I'll venture a wild guess that you probably haven't been working in the field as long as he has, and you don't personally know as many people in the field as he does, huh? Or is my presumptuous chain of logic just a random thoughtless unconscious probabilistic stream of tokens that has nothing to do with reality, and I don't realize I'm speaking to Ashish Vaswani himself? Then why don't you use your real name as you HN handle, are you really that modest, Ashish? And don't you have better things to do than posting to Hacker News? If you really wanted to win an argument from authority, you could at least post under your real name, Mr. Vaswani.

DonHopkins 1y ago

>"The quick brown fox jumped over the lazy ____". Correct prediction is "dogs". I would have predicted a single lazy "dog". Does that mean I'm more conscious than you? ;)

kazinator 1y ago
You don't get to pick what is "correct"; that's dictated by the training data. (Which cuold contain multiple versions that have either choice).
- DonHopkins 1y ago
  Neither do you, yet you literally just brashly and overconfidently hallucinated exactly like a poorly trained token predicting LLM that the: >Correct prediction is "dogs". Maybe I just trained on Wikipedia, or the February 9, 1885 edition of The Boston Journal "Current Notes" article, or Linda Bronson's 1888 book "Illustrative Shorthand", for what the statistically and historically "correct" completion is. https://en.wikipedia.org/wiki/The_quick_brown_fox_jumps_over_the_lazy_dog https://en.wikipedia.org/wiki/The_quick_brown_fox_jumps_over... [..._the_lazy_dog] Why would you insert redundant letters into a lovely well-known succinct pangram, when the whole point is to make the SHORTEST sentence with all letters of the alphabet? This entire post could also technically be considered a terribly redundant pangram if you wanted to throw in a lot of unnecessary letters, slashes, colons, underscores, double quotes, commas, periods, question marks, brackets, equal signs, digits, spaces, and newlines. As could Moby Dick, War and Peace, or the Unabomber Manifesto. >So actually it is like calling a grass-cutting machine "lawn mower". Is it anything like calling Larry Ellison a "lawn mower", which is much more "correct" than anthropomorphising him? https://news.ycombinator.com/item?id=15886728 https://news.ycombinator.com/item?id=15886728 As ninetyninenine pointed out, we don't understand what's going on with you either: >Someone can claim that the responses are an illusion and they have no meaning but that exact argument applies to humans and we can’t say anything either way. We don’t understand what’s going on.
  - kazinator 1y ago
    What I meant was that the correct prediction is "dogs" under the assumption that that is what has occurred in the arbitrarily selected training data. While we are training the thing, we want to get it to predict whatever is in the data. > Why would you insert redundant letters into a lovely well-known succinct pangram The 's' is not redundant, because my version of the pangram has "fox jumped" not "fox jumps". I picked up this version as a kid and have always used that one when testing new keyboards. > the whole point is to make the SHORTEST sentence with all letters of the alphabet If that is the whole point, the fox-dog sentence must be abandoned for a shorter pangram. The requirement those shorter pangrams tend not to satisfy is being easy to remember. People don't care about the sentence being short, which is why variants starting with "The ..." instead of "A ..." soon appeared after the invention of the original, which itself could have dropped the "A".