4 ms·

> they are not like humans at all, and only resemble them superficially due to our anthropomorphic tendency to infer a mind upon seeing fluent and roughly corre

by ta8645 3y ago

> they are not like humans at all, and only resemble them superficially due to our anthropomorphic tendency to infer a mind upon seeing fluent and roughly correspondent text. Alternatively, we overestimate human abilities, and imbue our thoughts with deep meaning and significance, simply as a way to feel more important than we actually are. Ultimately, we may rely on statistical prediction quite a bit as well. Obviously, LLMs aren't at the level of human cognition, but the impressive results they do produce, given their obvious simplicity, hints that intelligence may not be as hard to reproduce in silicon as we have previously assumed.

Jensson 3y ago
> Obviously, LLMs aren't at the level of human cognition, but the impressive results they do produce, given their obvious simplicity, hints that intelligence may not be as hard to reproduce in silicon as we have previously assumed. People said this about calculators as well, or initial chess AI as well. "If computers can do this, then the remaining parts of intelligence is probably not hard to reproduce!". So we already know many humans massively underestimate the depth of human intelligence. It happened many times before and will continue to happen every time we make computers do anything new.
- ta8645 3y ago
  > People said this about calculators as well, or initial chess AI as well. "If computers can do this, then the remaining parts of intelligence is probably not hard to reproduce!". People also said that chess computers would never beat the best humans. And then when that happened they said, "well they're just using sheer brute force, they will never produce beautiful chess". And then Alpha Zero came along, and showed that prediction was wrong as well. Personally, I believe that silicon intelligence will evolve a lot faster than ours did. But regardless, I've yet to hear any convincing argument why it can't happen eventually.
  - Jensson 3y ago
    > I've yet to hear any convincing argument why it can't happen eventually. That is a strawman, basically nobody argues this. > People also said that chess computers would never beat the best humans How many did say this? I see way more overly optimistic "humans are stupid" messages than I see "computers will never beat humans" messages. The argument is mostly between "humans are obsolete in a decade at most" versus "we would be lucky to see general intelligence in our lifetime", not that it it is impossible.
    - ta8645 3y ago
      > That is a strawman, basically nobody argues this. Some people do. Some people believe that intelligence is much more fundamental to the universe, and not capable of being reproduced mechanistically. But if you grant that it is possible, all we're talking about is when; which is anyone's guess. It could be tomorrow, it could take a million years. I'm not pretending to know, just offering a different perspective than the original comment.
      - Jensson 3y ago
        But those "some people" aren't here. What we are discussing is how close LLM are to human reasoning. Many argue LLM are already there, "what evidence do you have that humans aren't essentially just an LLM?" basically. You see such posts in this very thread.
    - mlyle 3y ago
      > That is a strawman, basically nobody argues this. I mean, people do-- even smart people like Penrose.
      - Jensson 3y ago
        But a large majority of people who say "LLMs are nothing like humans" are not making that argument. If you look through these kind of threads on HN you see maybe one post making that argument among hundreds, that is "basically nobody" to me given how often that strawman is used against people who are skeptical about LLMs potential capabilities.
- XorNot 3y ago
  This isn't a response to the OPs point, where they've stated their beliefs and their logic for them. To it, you've basically said: "well in that case defend the times some other people have been incorrect if you're sure this time". Why should they? The OP might be wrong, but whether they are or aren't has nothing to do with whether some unrelated, half-remembered third parties were right or wrong when mispredicting something else. You haven't addressed the content of the OP's actual argument at all, nor established a basis on which it may share similarity with other mispredictions.
  - Jensson 3y ago
    > but whether they are or aren't has nothing to do with whether some unrelated, half-remembered third parties were right or wrong Their argument was "Alternatively, we overestimate human abilities". Since he made that argument it is relevant to see if humans overestimated or underestimated human abilities in the past. My argument is that there is a large tendency for smart people and researchers to underestimate human intelligence, to assume it is easy to solve, we have seen that happen many times in the past. So since smart humans often underestimate human abilities, the "we overestimate human abilities" theory is probably not correct.
    - XorNot 3y ago
      Except that isn't true at all: the history of animal research is one of us assigning all human behavior as "uniquely human, requiring great human intelligence" and then discovering it's either replicated in detail in other "lower" animals, or replicated by animals which don't have nearly the apparent cognitive capacity for it[1]. We have an enormous history of vastly over-estimating the intelligence needed for some behaviors to manifest, with the more accurate conclusion being that actually we simply don't understand the nature of intelligence very well. [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5394149/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5394149/
      - Jensson 3y ago
        You suggest we under-estimate animal intelligence, which is the same argument I'm making. The reason we underestimate how hard human intelligence is to solve is that we underestimate the animal part of our reasoning, animals can't play chess or do math or write sentences, so when computers do that people assume they are smart like humans. But that assumption hinges on the animal part, like navigating a forest, identifying plants to eat etc, isn't a fundamental part of intelligence or that it is easy to solve in comparison. So yeah, underestimating animal intelligence is a large part of it, I agree with you.
Guvante 3y ago
Being able to mix what you have seen before isn't an accurate representation of any popular model for human intelligence to my knowledge. It is akin to a backronym "well that sounds similar so is the same" while ignoring the real differences.
- ta8645 3y ago
  > Being able to mix what you have seen before But humans clearly do that. It's not all we do, but we clearly do that. And given the impressive results that LLMs produce, my guess is it or something like it, will play a significant role in the AGI of the future.
  - XorNot 3y ago
    I'd say possibly it is all we do, we just have a lot more hidden inputs. i.e. we're all dealing with inputs from our physical bodies now, and can't not be. There's a whole bit of social advice about dealing with people which is "remember you may be encountering someone on the worst day of their life" (the idea being, 95+% of the time you shouldn't escalate, particularly with strangers). What I do now, and how I think about things is being affected in ways I know are directly related to blood sugar, fatigue, pain (i.e. my legs are a little uncomfortable in a way yours aren't, but I moved them just now so they feel better - all unique inputs to me, all always there). But the other problem with the notion is people thinking of the knowledge bounds these systems derive as being "limits" - they're not. The whole point of machine learning is they're not interpolating between known datapoints, but rather extracting the rules which fit those data points. So stay between the points, and sure - it's some type of complex interpolation of seen inputs. But you don't have to stay between those datapoints - you can move of either end and extrapolate new ones entirely.
  - Guvante 3y ago
    I would say LLM is at best a precursor for a memory system. But saying we almost have AGI because we have a template for HDDs isn't exactly representative IMHO.
- lern_too_spel 3y ago
ffwd 3y ago
To me predicting the next token is obviously not how humans think. If I ask you to envision a green triangle and a red square next to each other, and then swap the shapes but keep the colors in the same locations, and answer what color is the triangle now, you say the triangle is red, but you do so because you envisioned the triangle swapping places and did the mental steps etc. An LLM if even answering correctly, is statistically answering based on billions of lines of text + rlhf and all of this, I highly doubt there is a mental model of the world, but rather a large set of constraints in the probabilities which leads to the resulting answer. The reasoning ability is a secondary effect of the probabilities which is why it's hard to make it so every probability is correct for every answer I think. And regarding OP about hallucinating vs confabulating. To me hallucinating is a fine word for it, because it is filling in a gap or there aren't enough constraints in the model/data/tuning to account for that specific answer that it gave that was incorrect. Hence it "hallucinates" something in the gap. The real power of LLM's is that it seems to accumulate these 'constraints' (generalization), so that with the right model, it should be able to answer more and more prompts correctly, which is kind of amazing. Confabulation works too but is a little more high level IMO.
- klik99 3y ago
  I'm not entirely sure if the mental model is somehow a layer deeper than the prediction that humans do. I used to believe it, and still use it as shorthand, but these days I'm not sure it's accurate. The triangle example doesn't prove it because our predictive model could also say "hey, things don't just change color and shape like that, they need to move". It's similar to how an LLM can be more accurate with math when asked to step through and reason through it's logic - by stepping through the individual steps it can create a larger system. The thing that made me question if a "mental model" is at the base of human cognition - people who do those memory competitions, the clear winning strategy is the memory palace, or imagining walking through a house where each room is another number - they have to build step by step memory, it's not like an SQL database where they can just SELECT from random. Another one was the insight from GTD that if you remember 7 things to do, you're always repeating those 7 things to yourself to keep them in active memory. There's a strong argument that the mental model is derivative of a predictive model in the human brain, and we can just appear to have a mental model since we have an internal dialogue that runs so fast in the background that even we rarely recognize it. (anyone who has kept a steady meditation or similar practice should be familiar with it)

by ta8645 3y ago

Jensson 3y ago

> Obviously, LLMs aren't at the level of human cognition, but the impressive results they do produce, given their obvious simplicity, hints that intelligence may not be as hard to reproduce in silicon as we have previously assumed. People said this about calculators as well, or initial chess AI as well. "If computers can do this, then the remaining parts of intelligence is probably not hard to reproduce!". So we already know many humans massively underestimate the depth of human intelligence. It happened many times before and will continue to happen every time we make computers do anything new.

ta8645 3y ago
> People said this about calculators as well, or initial chess AI as well. "If computers can do this, then the remaining parts of intelligence is probably not hard to reproduce!". People also said that chess computers would never beat the best humans. And then when that happened they said, "well they're just using sheer brute force, they will never produce beautiful chess". And then Alpha Zero came along, and showed that prediction was wrong as well. Personally, I believe that silicon intelligence will evolve a lot faster than ours did. But regardless, I've yet to hear any convincing argument why it can't happen eventually.
- Jensson 3y ago
  > I've yet to hear any convincing argument why it can't happen eventually. That is a strawman, basically nobody argues this. > People also said that chess computers would never beat the best humans How many did say this? I see way more overly optimistic "humans are stupid" messages than I see "computers will never beat humans" messages. The argument is mostly between "humans are obsolete in a decade at most" versus "we would be lucky to see general intelligence in our lifetime", not that it it is impossible.
  - ta8645 3y ago
    > That is a strawman, basically nobody argues this. Some people do. Some people believe that intelligence is much more fundamental to the universe, and not capable of being reproduced mechanistically. But if you grant that it is possible, all we're talking about is when; which is anyone's guess. It could be tomorrow, it could take a million years. I'm not pretending to know, just offering a different perspective than the original comment.
    - Jensson 3y ago
      But those "some people" aren't here. What we are discussing is how close LLM are to human reasoning. Many argue LLM are already there, "what evidence do you have that humans aren't essentially just an LLM?" basically. You see such posts in this very thread.
  - mlyle 3y ago
    > That is a strawman, basically nobody argues this. I mean, people do-- even smart people like Penrose.
    - Jensson 3y ago
      But a large majority of people who say "LLMs are nothing like humans" are not making that argument. If you look through these kind of threads on HN you see maybe one post making that argument among hundreds, that is "basically nobody" to me given how often that strawman is used against people who are skeptical about LLMs potential capabilities.
XorNot 3y ago
This isn't a response to the OPs point, where they've stated their beliefs and their logic for them. To it, you've basically said: "well in that case defend the times some other people have been incorrect if you're sure this time". Why should they? The OP might be wrong, but whether they are or aren't has nothing to do with whether some unrelated, half-remembered third parties were right or wrong when mispredicting something else. You haven't addressed the content of the OP's actual argument at all, nor established a basis on which it may share similarity with other mispredictions.
- Jensson 3y ago
  > but whether they are or aren't has nothing to do with whether some unrelated, half-remembered third parties were right or wrong Their argument was "Alternatively, we overestimate human abilities". Since he made that argument it is relevant to see if humans overestimated or underestimated human abilities in the past. My argument is that there is a large tendency for smart people and researchers to underestimate human intelligence, to assume it is easy to solve, we have seen that happen many times in the past. So since smart humans often underestimate human abilities, the "we overestimate human abilities" theory is probably not correct.
  - XorNot 3y ago
    Except that isn't true at all: the history of animal research is one of us assigning all human behavior as "uniquely human, requiring great human intelligence" and then discovering it's either replicated in detail in other "lower" animals, or replicated by animals which don't have nearly the apparent cognitive capacity for it[1]. We have an enormous history of vastly over-estimating the intelligence needed for some behaviors to manifest, with the more accurate conclusion being that actually we simply don't understand the nature of intelligence very well. [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5394149/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5394149/
    - Jensson 3y ago
      You suggest we under-estimate animal intelligence, which is the same argument I'm making. The reason we underestimate how hard human intelligence is to solve is that we underestimate the animal part of our reasoning, animals can't play chess or do math or write sentences, so when computers do that people assume they are smart like humans. But that assumption hinges on the animal part, like navigating a forest, identifying plants to eat etc, isn't a fundamental part of intelligence or that it is easy to solve in comparison. So yeah, underestimating animal intelligence is a large part of it, I agree with you.

Guvante 3y ago

Being able to mix what you have seen before isn't an accurate representation of any popular model for human intelligence to my knowledge. It is akin to a backronym "well that sounds similar so is the same" while ignoring the real differences.

ta8645 3y ago
> Being able to mix what you have seen before But humans clearly do that. It's not all we do, but we clearly do that. And given the impressive results that LLMs produce, my guess is it or something like it, will play a significant role in the AGI of the future.
- XorNot 3y ago
  I'd say possibly it is all we do, we just have a lot more hidden inputs. i.e. we're all dealing with inputs from our physical bodies now, and can't not be. There's a whole bit of social advice about dealing with people which is "remember you may be encountering someone on the worst day of their life" (the idea being, 95+% of the time you shouldn't escalate, particularly with strangers). What I do now, and how I think about things is being affected in ways I know are directly related to blood sugar, fatigue, pain (i.e. my legs are a little uncomfortable in a way yours aren't, but I moved them just now so they feel better - all unique inputs to me, all always there). But the other problem with the notion is people thinking of the knowledge bounds these systems derive as being "limits" - they're not. The whole point of machine learning is they're not interpolating between known datapoints, but rather extracting the rules which fit those data points. So stay between the points, and sure - it's some type of complex interpolation of seen inputs. But you don't have to stay between those datapoints - you can move of either end and extrapolate new ones entirely.
- Guvante 3y ago
  I would say LLM is at best a precursor for a memory system. But saying we almost have AGI because we have a template for HDDs isn't exactly representative IMHO.
lern_too_spel 3y ago

ffwd 3y ago

To me predicting the next token is obviously not how humans think. If I ask you to envision a green triangle and a red square next to each other, and then swap the shapes but keep the colors in the same locations, and answer what color is the triangle now, you say the triangle is red, but you do so because you envisioned the triangle swapping places and did the mental steps etc. An LLM if even answering correctly, is statistically answering based on billions of lines of text + rlhf and all of this, I highly doubt there is a mental model of the world, but rather a large set of constraints in the probabilities which leads to the resulting answer. The reasoning ability is a secondary effect of the probabilities which is why it's hard to make it so every probability is correct for every answer I think. And regarding OP about hallucinating vs confabulating. To me hallucinating is a fine word for it, because it is filling in a gap or there aren't enough constraints in the model/data/tuning to account for that specific answer that it gave that was incorrect. Hence it "hallucinates" something in the gap. The real power of LLM's is that it seems to accumulate these 'constraints' (generalization), so that with the right model, it should be able to answer more and more prompts correctly, which is kind of amazing. Confabulation works too but is a little more high level IMO.

klik99 3y ago
I'm not entirely sure if the mental model is somehow a layer deeper than the prediction that humans do. I used to believe it, and still use it as shorthand, but these days I'm not sure it's accurate. The triangle example doesn't prove it because our predictive model could also say "hey, things don't just change color and shape like that, they need to move". It's similar to how an LLM can be more accurate with math when asked to step through and reason through it's logic - by stepping through the individual steps it can create a larger system. The thing that made me question if a "mental model" is at the base of human cognition - people who do those memory competitions, the clear winning strategy is the memory palace, or imagining walking through a house where each room is another number - they have to build step by step memory, it's not like an SQL database where they can just SELECT from random. Another one was the insight from GTD that if you remember 7 things to do, you're always repeating those 7 things to yourself to keep them in active memory. There's a strong argument that the mental model is derivative of a predictive model in the human brain, and we can just appear to have a mental model since we have an internal dialogue that runs so fast in the background that even we rarely recognize it. (anyone who has kept a steady meditation or similar practice should be familiar with it)