3 ms·

“It’s just auto complete” is probably one of the worst takes out there on AI. It’s also clear the author doesn’t actually know how LLMs work and is parroting i

“It’s just auto complete” is probably one of the worst takes out there on AI. It’s also clear the author doesn’t actually know how LLMs work and is parroting information, e.g. “…it tries to get you to finish your sentence with the statistically median thing that everyone would type next, on average.” is just not correct, and, frankly, suggests the author hasn’t even observed what autocomplete does. I completely respect the view that there’s more to being human than pattern matching text, but I also am open to the possibility consciousness may not actually be that much more than stochastic parrots.

textninja 4y ago
It’s a bad take for several reasons. - It’s reductive. The model may be architecturally relatively simple, at least insofar as humans are involved, but the data that was used train it is anything but. Code is data, data is code. The model is more than its architecture. It’s all the weights and biases within. - It’s qualitatively wrong. The temperature settings change it from a predictive system (auto complete) to a generative one (digital assistant). - It’s quantitatively ignorant. How do you eat an elephant? One bite at a time. For that matter, how do you speak if not one word at a time? It happens that the model generates one token at a time as well, but the million dollar question remains: what chain of reasoning did it have to follow to come up with that token? Just because it‘s only outputting a single token doesn’t mean it’s not “thinking” about the structure of the sentence, paragraph, or the entire document - several tokens ahead, to say the least. Indeed, having been trained on a such a large internet corpus it has the capacity to model entire personalities and all the quirks and neuroses that go on behind the scenes. So, yes, it is a bad take. That being said, it’s still a useful one because it encourages people to think about the models as tools, which is exactly correct given how they’re structured. So I still describe them as such.
- anileated 4y ago
  > “thinking” about the structure of the sentence, paragraph, or the entire document - several tokens ahead, to say the least You put thinking in quotes, so what exactly do you mean by thinking? Where does it do it? When? AI as autocomplete is a perfect summarization of both what it does and how it does it. Anything more, including models “thinking”, is strategic PR aimed to make the public happy with their mouths gaping while they take all intellectual property ever published online and monetize it without paying the author.
ieee2 4y ago
Agreed
pokeypokes 4y ago
"It's just a parrot" parroted repeatedly to suggest a lack of deeper understanding, while revealing the same.. So meta. The general reaction of people to all this has been one of the most interesting things about it imo. Earlier versions were generally just fun and sometimes amazing but suddenly so many are going out of their ways to play it down. Of course there are valid criticisms and worthy discussions but some of it feels like more than that. I think one side of it is that for the first time many people are genuinely comparing bots to humans, which by itself is kind of mind blowing. Another side seems to be more about "controlling" something new and scary. Maybe that thing is tangible, like the tools themselves, or maybe it's just the idea that we're not that special.

textninja 4y ago

It’s a bad take for several reasons. - It’s reductive. The model may be architecturally relatively simple, at least insofar as humans are involved, but the data that was used train it is anything but. Code is data, data is code. The model is more than its architecture. It’s all the weights and biases within. - It’s qualitatively wrong. The temperature settings change it from a predictive system (auto complete) to a generative one (digital assistant). - It’s quantitatively ignorant. How do you eat an elephant? One bite at a time. For that matter, how do you speak if not one word at a time? It happens that the model generates one token at a time as well, but the million dollar question remains: what chain of reasoning did it have to follow to come up with that token? Just because it‘s only outputting a single token doesn’t mean it’s not “thinking” about the structure of the sentence, paragraph, or the entire document - several tokens ahead, to say the least. Indeed, having been trained on a such a large internet corpus it has the capacity to model entire personalities and all the quirks and neuroses that go on behind the scenes. So, yes, it is a bad take. That being said, it’s still a useful one because it encourages people to think about the models as tools, which is exactly correct given how they’re structured. So I still describe them as such.

anileated 4y ago
> “thinking” about the structure of the sentence, paragraph, or the entire document - several tokens ahead, to say the least You put thinking in quotes, so what exactly do you mean by thinking? Where does it do it? When? AI as autocomplete is a perfect summarization of both what it does and how it does it. Anything more, including models “thinking”, is strategic PR aimed to make the public happy with their mouths gaping while they take all intellectual property ever published online and monetize it without paying the author.

ieee2 4y ago

Agreed

pokeypokes 4y ago

"It's just a parrot" parroted repeatedly to suggest a lack of deeper understanding, while revealing the same.. So meta. The general reaction of people to all this has been one of the most interesting things about it imo. Earlier versions were generally just fun and sometimes amazing but suddenly so many are going out of their ways to play it down. Of course there are valid criticisms and worthy discussions but some of it feels like more than that. I think one side of it is that for the first time many people are genuinely comparing bots to humans, which by itself is kind of mind blowing. Another side seems to be more about "controlling" something new and scary. Maybe that thing is tangible, like the tools themselves, or maybe it's just the idea that we're not that special.