7 ms·

After using them for a long time I am convinced they have no true intelligence beyond what is latent in training data. In other words I think we are kind of foo

by api 2y ago

After using them for a long time I am convinced they have no true intelligence beyond what is latent in training data. In other words I think we are kind of fooling ourselves. That being said they are very useful. I mostly use them as a far superior alternative to web search and as a kind of junior research assistant. Anything they find must be checked of course. I think we have invented the sci-fi trope of the AI librarian of the galactic archive. It can’t solve problems but it can rifle through the totality of human knowledge and rapidly find things. It’s a search engine.

weakfish 2y ago
I mean, it’s known that there’s no intelligence if you simply look at how it works on a technical level - it’s a prediction of the next token. That wasn’t really ever in question as to whether they have “intelligence”
- api 2y ago
  To people who really understand them and are grounded, I think you're right. There has been a lot of hype among people who don't understand them as much, a lot of hype among the public, and a lot of schlock about "superintelligence" and "hard takeoff" etc. among smart but un-grounded people. The latter kind of fear mongering hype has been exploited by companies like ClosedAI in a bid for regulatory capture.
  - danielbln 2y ago
    A little humility would do us good regardless, because we don't know what intelligence is and what consciousness is, we can't truly define it nor do we understand what makes humans conscious and sentient/sapient. Categorically ruling out intelligence because "it's just a token predictor" puts us at the opposite of the spectrum, and that's not necessarily a better place to be.
- jghn 2y ago
  > it’s known that there’s no intelligence To you & I that's true. But especially for the masses that's not true. It seems like at least once to day that I either talk to someone or hear someone via tv/radio/etc who does not understand this. An example that amused me recently was a radio talk show host who had a long segment describing how he & a colleague had a long argument with ChatGPT to correct a factual inaccuracy about their radio show. And that they finally convinced ChatGPT that they were correct due to their careful use of evidence & reasoning. And the part they were most happy about was how it had now learned, and going forward ChatGPT would not spread these inaccuracies. That anecdote is how the public at large sees these tools.
  - dylan604 2y ago
    > radio talk show Well, there's the first problem. > were most happy about was how it had now learned on tomorrow's episode, those same hosts learn that once their chat session ended, the same conversion gets to start all over from the beginning.
  - ithkuil 2y ago
    Ironically if you explain to those talk show hosts how they are wrong about how ChatGPT learns (or doesn't learn) and use all the right arguments and proofs so that they finally concede, chances are that they too won't quite learn from that and keep repeating their previous bias next time.
  - weakfish 2y ago
    Oh I totally agree, it bugs me to no end and that’s partially why I replied :)
returnInfinity 2y ago
But Ilya has convinced himself and many others that predicting the next token is intelligence.
- swid 2y ago
  It seems to me predicting things in general is a pretty good way to bootstrap intelligence. If you are competing for resources, predicting how to avoid danger and catch food is about the most basic way to reinforce good behavior.
  - api 2y ago
    This would fall down into a semantic debate over what is meant by intelligence. There is a well known phenomenon known as the AI effect: when something works we start calling it something else, not AI. Heuristics and complex reasoning trees were once called AI. Fuzzy logic with control systems was once called AI. Clustering was once called AI. And so on… This certainly has one root in human or carbon-based life cheuvanism but I think there’s something essential happening too. With each innovation we see its limits and it causes us to go back and realize that what we colloquially call intelligence was more than we thought it was. Intelligence predicts, but is prediction intelligence? Again, here by intelligence I mean what complex living organisms and humans do. I still believe there are things going on here not modeled by any CS system and not well understood. Not magic, just not solved yet. We are reverse engineering billions of years of evolution folks. We won’t figure it all out in a few decades.
    - swid 2y ago
      Demonstrably, humans do think, and arguably demonstrably, early life would go down a path of simple predictions (in the form of stimulus -> response). And demonstrably, evolution did lead to human level intelligence. So I don’t think there needs to be a semantic debate over where in the process intelligence started. The early responses to stimulus is a form of prediction, but not one that requires thinking. There can be much disagreement that prediction is at the core of intelligence, or if optimizing ability to predict leads to intelligence. But from the established facts, it is the case the higher forms of life were bootstrapped from the lower ones, and also our biochemistry does have reward functions. Successfully triggering those rewards will generally hinge on making successful predictions. Take from that what you will.
zcw100 2y ago
And a plagiarism machine. It's like a high school student that thinks they can change a couple of words, make sure it's grammatically correct and it's not plagiarism because it's not an exact quote....either that or it just completely makes it up. I think LLMs will be revolutionary but just not in the way people think. It may be similar to the Gutenberg press. Before the printing press words were precious and closely held resources. The Gutenberg press made words cheap and abundant. Not everyone thought it was a good thing at the time but it changed everything.

by api 2y ago

weakfish 2y ago

I mean, it’s known that there’s no intelligence if you simply look at how it works on a technical level - it’s a prediction of the next token. That wasn’t really ever in question as to whether they have “intelligence”

api 2y ago
To people who really understand them and are grounded, I think you're right. There has been a lot of hype among people who don't understand them as much, a lot of hype among the public, and a lot of schlock about "superintelligence" and "hard takeoff" etc. among smart but un-grounded people. The latter kind of fear mongering hype has been exploited by companies like ClosedAI in a bid for regulatory capture.
- danielbln 2y ago
  A little humility would do us good regardless, because we don't know what intelligence is and what consciousness is, we can't truly define it nor do we understand what makes humans conscious and sentient/sapient. Categorically ruling out intelligence because "it's just a token predictor" puts us at the opposite of the spectrum, and that's not necessarily a better place to be.
jghn 2y ago
> it’s known that there’s no intelligence To you & I that's true. But especially for the masses that's not true. It seems like at least once to day that I either talk to someone or hear someone via tv/radio/etc who does not understand this. An example that amused me recently was a radio talk show host who had a long segment describing how he & a colleague had a long argument with ChatGPT to correct a factual inaccuracy about their radio show. And that they finally convinced ChatGPT that they were correct due to their careful use of evidence & reasoning. And the part they were most happy about was how it had now learned, and going forward ChatGPT would not spread these inaccuracies. That anecdote is how the public at large sees these tools.
- dylan604 2y ago
  > radio talk show Well, there's the first problem. > were most happy about was how it had now learned on tomorrow's episode, those same hosts learn that once their chat session ended, the same conversion gets to start all over from the beginning.
- ithkuil 2y ago
  Ironically if you explain to those talk show hosts how they are wrong about how ChatGPT learns (or doesn't learn) and use all the right arguments and proofs so that they finally concede, chances are that they too won't quite learn from that and keep repeating their previous bias next time.
- weakfish 2y ago
  Oh I totally agree, it bugs me to no end and that’s partially why I replied :)

returnInfinity 2y ago

But Ilya has convinced himself and many others that predicting the next token is intelligence.

swid 2y ago
It seems to me predicting things in general is a pretty good way to bootstrap intelligence. If you are competing for resources, predicting how to avoid danger and catch food is about the most basic way to reinforce good behavior.
- api 2y ago
  This would fall down into a semantic debate over what is meant by intelligence. There is a well known phenomenon known as the AI effect: when something works we start calling it something else, not AI. Heuristics and complex reasoning trees were once called AI. Fuzzy logic with control systems was once called AI. Clustering was once called AI. And so on… This certainly has one root in human or carbon-based life cheuvanism but I think there’s something essential happening too. With each innovation we see its limits and it causes us to go back and realize that what we colloquially call intelligence was more than we thought it was. Intelligence predicts, but is prediction intelligence? Again, here by intelligence I mean what complex living organisms and humans do. I still believe there are things going on here not modeled by any CS system and not well understood. Not magic, just not solved yet. We are reverse engineering billions of years of evolution folks. We won’t figure it all out in a few decades.
  - swid 2y ago
    Demonstrably, humans do think, and arguably demonstrably, early life would go down a path of simple predictions (in the form of stimulus -> response). And demonstrably, evolution did lead to human level intelligence. So I don’t think there needs to be a semantic debate over where in the process intelligence started. The early responses to stimulus is a form of prediction, but not one that requires thinking. There can be much disagreement that prediction is at the core of intelligence, or if optimizing ability to predict leads to intelligence. But from the established facts, it is the case the higher forms of life were bootstrapped from the lower ones, and also our biochemistry does have reward functions. Successfully triggering those rewards will generally hinge on making successful predictions. Take from that what you will.

zcw100 2y ago

And a plagiarism machine. It's like a high school student that thinks they can change a couple of words, make sure it's grammatically correct and it's not plagiarism because it's not an exact quote....either that or it just completely makes it up. I think LLMs will be revolutionary but just not in the way people think. It may be similar to the Gutenberg press. Before the printing press words were precious and closely held resources. The Gutenberg press made words cheap and abundant. Not everyone thought it was a good thing at the time but it changed everything.