3 ms·

The amount of anthropomorphizing of these LLMs in this thread is off the charts. These language models do not have human intelligence, nor do they approximate i

by pyth0 3y ago

The amount of anthropomorphizing of these LLMs in this thread is off the charts. These language models do not have human intelligence, nor do they approximate it, though they do an incredible job at mimicking what the result of intelligence looks like. They are susceptible to prompt injection precisely because of this, and it is why I don't know if it can ever be 100% solved with these models.

thornewolf 3y ago
"It merely has all of the byproducts of intelligence, its not intelligence though!" I make this statement in a frank way to rhetorically get the point across. I find myself continually surprised by the general community's desire to reject the intelligence claim in its entirely. I make no claim that this intelligence manifest in the same way human intelligence does. I make no claim that this intelligence can even be measured in the same way a humans intelligence does. What I do claim though is that it is intelligence - intelligence that relates to humans in the same way the mind of a crow might. The dominant mindset I have observed in my life thus-far when people discuss human intelligence is the pattern matching perspective. Humans are differentiates by our outsized ability to pattern match being able to successfully manipulate these patterns. We now see something nonorganic with amazing pattern matching abilities. We have previously seen other organic entities with impressive pattern matching abilities. Why must this situation be any different? My overall claims: - Intelligence is best measured by outcomes. How some entity is best able to manipulate its existence (however that existence may manifest) - Intelligence can manifest in more than one way. An entirely mechanical system could be considered to have some level of "intelligence" - Considering something intelligent or to have desires is not anthropomorphizing. There are many non-human entities that we consider to have these properties.
- pyth0 3y ago
  > Intelligence is best measured by outcomes. How some entity is best able to manipulate its existence. I agree, although I think we humans have always been fairly bad at measuring intelligence in a way that truly appreciates all the complexity of it. The second part of that is also interesting and I would agree that is partly what makes these LLMs non-intelligent. The models do not really have "an existence" outside of the moment in which they are processing the context and producing output. > Intelligence can manifest in more than one way. An entirely mechanical system could be considered to have some level of "intelligence" I don't think I agree with this, or at least maybe I disagree with your definition of "intelligent". I believe that intelligence is heavily intertwined with biology and it exists is all manner of non-human creatures but I don't think I would call an entirely mechanical system "intelligent". Perhaps I would say it had "intelligent design". > Considering something intelligent or to have desires is not anthropomorphizing. I absolutely agree with this and I was not trying to imply that it was unique to humans. In fact I think we severely discount the amount of intelligence in non-human life forms all the time. I do think that ChatGPT possesses knowledge (as encoded in its weights) similar to a book, however unlike a book it also has a convenient and familiar interface that allows us to interact with this knowledge and form unique and novel results.
  - stevenhuang 3y ago
    Category error. You should look up the definition of intelligence. Your definition is unorthodox. Intelligence is a much more encompassing term that describes abstract reasoning ability among others. All biological systems are intelligent to varying degrees, but not all intelligent systems are biological.
  - jimbokun 3y ago
    You can define “intelligence” to only refer to biological intelligence. But that doesn’t mean that AIs can’t do things we call intelligent in humans at or beyond a human level.
- scott_w 3y ago
  Because while it’s mimicking a human kind of intelligence, it’s missing the kinds of intelligence that even basic mammals have. One example: it has no concept of objects and permanence. Something even my dog has. Want an example? Watch Gotham Chess on YouTube play it at chess and you’ll see it not only doesn’t understand the rules of the game, it can’t even remember which pieces are on the board!
  - dj_mc_merlin 3y ago
    Dogs can't use complex language at all, so it's also missing a kind of intelligence that the model has. It is not surprising that a pattern matching device tuned to just text (and some 2d images I believe?) doesn't have a great understanding of concepts that are obvious in the physical world. It's more surprising that it is often able to approximate pretty well without having any first hand data about it.
    - scott_w 3y ago
      You’re right, dogs can’t do that. That’s because intelligence is clearly a multi faceted and extremely complex concept to define. It’s so difficult to define, in fact, that it seems we’re only able to do so by pointing at things and going “that’s not it.” That’s not moving the goalposts at all. If it were, we’d have stopped at search algorithms back in the 60s and declared AI to be “solved.” You mention the physical world, so let’s talk about self driving cars, the last thing we thought would be “AI” just a decade ago. It’s 2023, and Tesla still can’t stop their cars driving into concrete barriers. Something that the system was built for and, again, pretty much every animal can do without thinking. All of these systems and research definitely get us closer to understanding intelligence (and maybe creating AI one day) but to say they are intelligent is to ignore your own intelligence that knows they obviously are not.
scarface_74 3y ago
This type of comment sounds like the one that comes up anytime someone mentions “serverless”. “Well there is no such thing as serverless. There are servers in the background”. Yes people on HN already know that. We also know that Alice and Bob are not real people working in cryptography.
- pyth0 3y ago
  Yeah no I don't believe this is a fair comparison at all, and I'm frankly surprised you think this is accurate to the discussion around LLMs. There are certainly people on here who believe and talk about ChatGPT as if it is generally intelligent. I suppose if you really want I can look through previous threads, but you really can find this under most threads about ChatGPT. A brand of this fallacious reasoning I find particularly annoying are responses that take the form of "well humans also do <reductive vague parallel to LLM operation>" usually in response to people pointing out weaknesses in these language models. It doesn't really matter whether these commenters believe it or not, it does not further the discussion in a meaningful way and it perpetuates FUD around the "AI takeover".
permo-w 3y ago
on a technical level, can you explain the difference between pre-trained transformers and human language processing? why does this difference make them more susceptible to prompt injections than we are to—say—lies? I’m not saying you’re wrong, I just want to see your working

by pyth0 3y ago

thornewolf 3y ago

"It merely has all of the byproducts of intelligence, its not intelligence though!" I make this statement in a frank way to rhetorically get the point across. I find myself continually surprised by the general community's desire to reject the intelligence claim in its entirely. I make no claim that this intelligence manifest in the same way human intelligence does. I make no claim that this intelligence can even be measured in the same way a humans intelligence does. What I do claim though is that it is intelligence - intelligence that relates to humans in the same way the mind of a crow might. The dominant mindset I have observed in my life thus-far when people discuss human intelligence is the pattern matching perspective. Humans are differentiates by our outsized ability to pattern match being able to successfully manipulate these patterns. We now see something nonorganic with amazing pattern matching abilities. We have previously seen other organic entities with impressive pattern matching abilities. Why must this situation be any different? My overall claims: - Intelligence is best measured by outcomes. How some entity is best able to manipulate its existence (however that existence may manifest) - Intelligence can manifest in more than one way. An entirely mechanical system could be considered to have some level of "intelligence" - Considering something intelligent or to have desires is not anthropomorphizing. There are many non-human entities that we consider to have these properties.

pyth0 3y ago
> Intelligence is best measured by outcomes. How some entity is best able to manipulate its existence. I agree, although I think we humans have always been fairly bad at measuring intelligence in a way that truly appreciates all the complexity of it. The second part of that is also interesting and I would agree that is partly what makes these LLMs non-intelligent. The models do not really have "an existence" outside of the moment in which they are processing the context and producing output. > Intelligence can manifest in more than one way. An entirely mechanical system could be considered to have some level of "intelligence" I don't think I agree with this, or at least maybe I disagree with your definition of "intelligent". I believe that intelligence is heavily intertwined with biology and it exists is all manner of non-human creatures but I don't think I would call an entirely mechanical system "intelligent". Perhaps I would say it had "intelligent design". > Considering something intelligent or to have desires is not anthropomorphizing. I absolutely agree with this and I was not trying to imply that it was unique to humans. In fact I think we severely discount the amount of intelligence in non-human life forms all the time. I do think that ChatGPT possesses knowledge (as encoded in its weights) similar to a book, however unlike a book it also has a convenient and familiar interface that allows us to interact with this knowledge and form unique and novel results.
- stevenhuang 3y ago
  Category error. You should look up the definition of intelligence. Your definition is unorthodox. Intelligence is a much more encompassing term that describes abstract reasoning ability among others. All biological systems are intelligent to varying degrees, but not all intelligent systems are biological.
- jimbokun 3y ago
  You can define “intelligence” to only refer to biological intelligence. But that doesn’t mean that AIs can’t do things we call intelligent in humans at or beyond a human level.
scott_w 3y ago
Because while it’s mimicking a human kind of intelligence, it’s missing the kinds of intelligence that even basic mammals have. One example: it has no concept of objects and permanence. Something even my dog has. Want an example? Watch Gotham Chess on YouTube play it at chess and you’ll see it not only doesn’t understand the rules of the game, it can’t even remember which pieces are on the board!
- dj_mc_merlin 3y ago
  Dogs can't use complex language at all, so it's also missing a kind of intelligence that the model has. It is not surprising that a pattern matching device tuned to just text (and some 2d images I believe?) doesn't have a great understanding of concepts that are obvious in the physical world. It's more surprising that it is often able to approximate pretty well without having any first hand data about it.
  - scott_w 3y ago
    You’re right, dogs can’t do that. That’s because intelligence is clearly a multi faceted and extremely complex concept to define. It’s so difficult to define, in fact, that it seems we’re only able to do so by pointing at things and going “that’s not it.” That’s not moving the goalposts at all. If it were, we’d have stopped at search algorithms back in the 60s and declared AI to be “solved.” You mention the physical world, so let’s talk about self driving cars, the last thing we thought would be “AI” just a decade ago. It’s 2023, and Tesla still can’t stop their cars driving into concrete barriers. Something that the system was built for and, again, pretty much every animal can do without thinking. All of these systems and research definitely get us closer to understanding intelligence (and maybe creating AI one day) but to say they are intelligent is to ignore your own intelligence that knows they obviously are not.

scarface_74 3y ago

This type of comment sounds like the one that comes up anytime someone mentions “serverless”. “Well there is no such thing as serverless. There are servers in the background”. Yes people on HN already know that. We also know that Alice and Bob are not real people working in cryptography.

pyth0 3y ago
Yeah no I don't believe this is a fair comparison at all, and I'm frankly surprised you think this is accurate to the discussion around LLMs. There are certainly people on here who believe and talk about ChatGPT as if it is generally intelligent. I suppose if you really want I can look through previous threads, but you really can find this under most threads about ChatGPT. A brand of this fallacious reasoning I find particularly annoying are responses that take the form of "well humans also do <reductive vague parallel to LLM operation>" usually in response to people pointing out weaknesses in these language models. It doesn't really matter whether these commenters believe it or not, it does not further the discussion in a meaningful way and it perpetuates FUD around the "AI takeover".

permo-w 3y ago

on a technical level, can you explain the difference between pre-trained transformers and human language processing? why does this difference make them more susceptible to prompt injections than we are to—say—lies? I’m not saying you’re wrong, I just want to see your working