4 ms·

The crazy thing is that people think that a model designed to predict sequences of tokens from a stem, no matter how advanced the model, to be much more than ju

by jdiff 1y ago

The crazy thing is that people think that a model designed to predict sequences of tokens from a stem, no matter how advanced the model, to be much more than just "really good autocomplete." It is impressive and very unintuitive just how far that can get you, but it's not reductive to use that label. That's what it is on a fundamental level, and aligning your usage with that will allow it to be more effective.

fl7305 1y ago
> "The crazy thing is that people think that a model designed to" It's even crazier that some people believe that humans "evolved" intelligence just by nature selecting the genes which were best at propagating. Clearly, human intelligence is the product of a higher being designing it. /s
- fhd2 1y ago
  I would consider evolution a form of intelligence, even though I wouldn't consider nature a being. There's a branch of AI research I was briefly working in 15 years ago, based on that premise: Genetic algorithms/programming. So I'd argue humans were (and are continuously being) designed, in a way.
  - fl7305 1y ago
    (non-sarcastically from me this time) Sure, I would agree with that wording. In the same way, neural networks which are trained to do a task could be said to be "designed" to do something. In my view, there's a big difference in what the training data is for a neural network, and what the neural network is "designed" for. We can train a network using word completion examples, with the intent of designing it for intelligence.
    - fhd2 1y ago
      Yup. To counter my own points a bit: I could also argue that the word "design" has a connotation strictly opposing emergent behaviour like evolution, as in the intelligent design "theory". So not the best word to use perhaps. And in your example, just because we made a system that exhibits emergent behaviour to some degree, we can't assume it can "design" intelligence the way evolution did, on a much, much shorter timeline, no less.
vidarh 1y ago
It's trivial to demonstrate that it takes only a tiny LLM + a loop to a have a Turing complete system. The extension of that is that it is utterly crazy to think that the fact it is "a model designed to predict sequences of tokens" puts much of a limitation on what an LLM can achieve - any Turing complete system can by definition simulate any other. To the extent LLMs are limited, they are limited by training and compute. But these endless claims that the fact they're "just" predicting tokens means something about their computational power are based on flawed assumptions.
- suddenlybananas 1y ago
  The fact they're Turing complete isn't really getting at the heart of the problem. Python is Turing complete and calling python "intelligent" would be a category error.
  - vidarh 1y ago
    It is getting to the heart of the problem when the claim made is that "no matter how advanced the model" they can't be 'much more than just "really good autocomplete."'. Given that they are Turing complete when you put a loop around them, that claim is objectively false.
    - jdiff 1y ago
      I think it'd even be easier to coerce standard autocomplete into demonstrating Turing completeness. And without burning millions of dollars of GPU hours on training it.
- msgodel 1y ago
  Language models with a loop absolutely aren't Turing complete. Assuming the model can even follow your instructions the output is probabilistic so in the limit you can guarantee failure. In reality though there are lots of instructions LLMs fail to follow. You don't notice it as much when you're using them normally but if you want to talk about computation you'll run into trivial failures all the time. The last time I had this discussion with people I pointed out how LLMs consistently and completely fail at applying grammar production rules (obviously you tell them to apply to words and not single letters so you don't fight with the embedding.) LLMs do some amazing stuff but at the end of the day: 1) They're just language models, while many things can be described with languages there are some things that idea doesn't capture. Namely languages that aren't modeled, which is the whole point of a Turing machine. 2) They're not human, and the value is always going to come from human socialization.
lavelganzu 1y ago
There's a plausible argument for it, so it's not a crazy thing. You as a human being can also predict likely completions of partial sentences, or likely lines of code given surrounding lines of code, or similar tasks. You do this by having some understanding of what the words mean and what the purpose of the sentence/code is likely to be. Your understanding is encoded in connections between neurons. So the argument goes: LLMs were trained to predict the next token, and the most general solution to do this successfully is by encoding real understanding of the semantics.
dwaltrip 1y ago
It’s reductive and misleading because autocomplete, as it’s commonly known, existed for many years before generative AI, and is very different and quite dumber than LLMs.

by jdiff 1y ago

fl7305 1y ago

> "The crazy thing is that people think that a model designed to" It's even crazier that some people believe that humans "evolved" intelligence just by nature selecting the genes which were best at propagating. Clearly, human intelligence is the product of a higher being designing it. /s

fhd2 1y ago
I would consider evolution a form of intelligence, even though I wouldn't consider nature a being. There's a branch of AI research I was briefly working in 15 years ago, based on that premise: Genetic algorithms/programming. So I'd argue humans were (and are continuously being) designed, in a way.
- fl7305 1y ago
  (non-sarcastically from me this time) Sure, I would agree with that wording. In the same way, neural networks which are trained to do a task could be said to be "designed" to do something. In my view, there's a big difference in what the training data is for a neural network, and what the neural network is "designed" for. We can train a network using word completion examples, with the intent of designing it for intelligence.
  - fhd2 1y ago
    Yup. To counter my own points a bit: I could also argue that the word "design" has a connotation strictly opposing emergent behaviour like evolution, as in the intelligent design "theory". So not the best word to use perhaps. And in your example, just because we made a system that exhibits emergent behaviour to some degree, we can't assume it can "design" intelligence the way evolution did, on a much, much shorter timeline, no less.

vidarh 1y ago

It's trivial to demonstrate that it takes only a tiny LLM + a loop to a have a Turing complete system. The extension of that is that it is utterly crazy to think that the fact it is "a model designed to predict sequences of tokens" puts much of a limitation on what an LLM can achieve - any Turing complete system can by definition simulate any other. To the extent LLMs are limited, they are limited by training and compute. But these endless claims that the fact they're "just" predicting tokens means something about their computational power are based on flawed assumptions.

suddenlybananas 1y ago
The fact they're Turing complete isn't really getting at the heart of the problem. Python is Turing complete and calling python "intelligent" would be a category error.
- vidarh 1y ago
  It is getting to the heart of the problem when the claim made is that "no matter how advanced the model" they can't be 'much more than just "really good autocomplete."'. Given that they are Turing complete when you put a loop around them, that claim is objectively false.
  - jdiff 1y ago
    I think it'd even be easier to coerce standard autocomplete into demonstrating Turing completeness. And without burning millions of dollars of GPU hours on training it.
msgodel 1y ago
Language models with a loop absolutely aren't Turing complete. Assuming the model can even follow your instructions the output is probabilistic so in the limit you can guarantee failure. In reality though there are lots of instructions LLMs fail to follow. You don't notice it as much when you're using them normally but if you want to talk about computation you'll run into trivial failures all the time. The last time I had this discussion with people I pointed out how LLMs consistently and completely fail at applying grammar production rules (obviously you tell them to apply to words and not single letters so you don't fight with the embedding.) LLMs do some amazing stuff but at the end of the day: 1) They're just language models, while many things can be described with languages there are some things that idea doesn't capture. Namely languages that aren't modeled, which is the whole point of a Turing machine. 2) They're not human, and the value is always going to come from human socialization.

lavelganzu 1y ago

There's a plausible argument for it, so it's not a crazy thing. You as a human being can also predict likely completions of partial sentences, or likely lines of code given surrounding lines of code, or similar tasks. You do this by having some understanding of what the words mean and what the purpose of the sentence/code is likely to be. Your understanding is encoded in connections between neurons. So the argument goes: LLMs were trained to predict the next token, and the most general solution to do this successfully is by encoding real understanding of the semantics.

dwaltrip 1y ago

It’s reductive and misleading because autocomplete, as it’s commonly known, existed for many years before generative AI, and is very different and quite dumber than LLMs.