5 ms·

People seem so convinced of this and I just don't get it. I'm seeing this comment through my eyeballs, generating some pertinent text in my brain, and outputtin

by iameli 4y ago

People seem so convinced of this and I just don't get it. I'm seeing this comment through my eyeballs, generating some pertinent text in my brain, and outputting it back out. But so many people seem convinced convinced this process is something radically, fundamentally, irreducibly different than what ChatGPT is doing internally, and I don't get why. Is it because I have a consciousness with an internal narrative and ChatGPT does not? Because that seems like more of a result of how we've wired up ChatGPT to operate than a fundamental structural difference; nothing stopping us from making ChatGPT talk to itself in its brain to generate synthesis.

JohnFen 4y ago
It's because whatever you and I are doing, it's not simply statistical analysis. That is all that ChatGPT is doing. It may be possible to create a machine that flirts with actual intelligence, but this is simply not it. There's not even room for doubt about this.
- iameli 4y ago
  I can't claim to understand everything my brain is doing, but accepting input and filtering it through a bunch of neuron chains to result in some kind of output sounds like "statistical analysis" to me. You seem convinced that ChatGPT will never have "actual intelligence" — care to make a prediction about something that LLMs will never accomplish? We know they can write code, write essays, generate artwork, and play chess. What's a task that requires "actual intelligence"? Parenting a child? Running for President? Making a steak sandwich?
  - JohnFen 4y ago
    An LLM will never be able to produce something that cannot be statistically derived from its training data. So... creating a new language, maybe?
    - iameli 4y ago
      ChatGPT's attempt: https://gist.github.com/iameli/d9a5b715ec9baa5b11063888e054d36a https://gist.github.com/iameli/d9a5b715ec9baa5b11063888e054d... Are those two responses enough to say it's invented a language? Probably not. But it's already farther along than I would have gotten if you asked me to do such a thing. If I kept prompting it for hours, I bet it would start to contradict itself and lose track of the rules it had already established, but so would I. If I were actually inventing a language, I'd take months and keep extensive cross-referenced documentation on grammar and syntax and vocabulary. We don't _let_ ChatGPT do that, it has no mechanism for persisting its ideas like that. But like... neither would I if you took away my notebook and the parts of my brain that persist long-term memory. I guess my interpretation here - I see differences in _capabilities_, but not differences in actual _intelligence_.
    - neongreen 4y ago
      https://maximumeffort.substack.com/p/i-taught-chatgpt-to-invent-a-language https://maximumeffort.substack.com/p/i-taught-chatgpt-to-inv... My bet would be that in ten years GPT would get pretty good at it, but eh, I dunno.
    - jesterman 4y ago
      I see what you are going with here for it not being able to come up with original things. But I think that falls apart when you realize almost nothing humans do is original, everything is just building on other things.
      - JohnFen 4y ago
        The vast majority of what people do is this, yes. Just like how most of the time, we're all running on autopilot and our behaviors are pretty much just following scripts. "Meat robots". But it's not 100%.
- School-Cotton 4y ago
  > whatever you and I are doing, it's not simply statistical analysis. How do you know?
  - JohnFen 4y ago
    Because I have seen genuine creativity and invention.
- deleted 4y ago
  [deleted]
- deleted 4y ago
  [deleted]
lapinot 4y ago
LLM (and deep learning in general) are to AGI what bogosort is to sorting. For some reason beyond me people think it's very important to not try to understand anything about the structure of the problem you're trying to solve and just make a very general algorithm which will be suboptimal in just about every way except for the generality of it's code. Seeing the world as a trillion dimensional token soup is definitely quite general and at the same time very very weak in terms of expressivity.
- iameli 4y ago
  Sure, I think buy this interpretation. As long as we can agree that bogosort and quicksort are both still sorting algorithms. My brain definitely has some structures that are useful for me understanding the world that are more immediately useful than trillion dimensional token soup. But I'm also not convinced that it's impossible those structures could be successfully emulated by quadrillion dimensional token soup. And a lot of folks seem to be convinced that it's some kind of fundamental impossibility.
- canjobear 4y ago
  No other approach has worked as well for natural language, and not for lack of trying.
hathawsh 4y ago
It's because you and I also have mental resources that are structured very differently from ChatGPT. Even if some part of our brain might resemble a LLM, a LLM is a very poor representation of other parts of the brain. If you take ChatGPT too far, it falls into repetition and demonstrates that it obviously has no comprehension of the material it's remixing. Maybe some combination of a LLM and other mental machinery would result in AGI with real comprehension.
- iameli 4y ago
  I dunno - I hear what you're saying about how it seems to lose track of things over long conversations. But part of the mental machinery that it lacks is the ability to learn from those conversations and insert new ideas into its base model; every new conversation starts fresh. It has no long-term storage; it can't "keep notes" for itself, and we don't give it the ability to alter its fundamental state based on new input after we're done feeding it training data. I guess we're mostly agreeing, because those things are probably part of the "other mental machinery" we'd want to provide it, but I'd push back a little on "real comprehension". It kinda seems to me like the "comprehension engine" is working just fine, it's the structure we've built around that comprehension engine that's limiting it right now.
dinkumthinkum 4y ago
You’re comparing a LLM to yourself, really? I wonder how many people talking about this know much about computers. Your cognitive capability is much more complex than ChatGPT. I’m not sure why people are convinced that a Turing Machine can simulate it, let alone do it efficiently. Do you believe that the way you think is at all like ChatGPT. That you are both processing text is not evidence. How many watts of power did you utilize on your whole life up to this point? How much did it take to train ChatGPT and how much does it take to run it, even if you could run it on your home PC? How much heat does it generate. You may not like it but could live and solve intellectual problems for a week on only water and a handful of rice. Do you really think these two things, human brains and computers based on TMs are the same type of thing?
- iameli 4y ago
  Your argument is that my brain is more energy-efficient than ChatGPT, therefore... ChatGPT is incapable of "understanding"?
  - dinkumthinkum 4y ago
    I’m not arguing that it is more energy efficient. I’m showing the obvious fact that there is a massive scale difference in the amount of energy and that one is capable of much more complex problem solving which seems to indicate that the two things are fundamentally different.
    - iameli 4y ago
      Okay. Same question I asked elsewhere in this thread; care to make a prediction about a problem that won't ever get solved by an LLM? We know they can write essays, create art, write code — what's something they'll never do?

by iameli 4y ago

JohnFen 4y ago

It's because whatever you and I are doing, it's not simply statistical analysis. That is all that ChatGPT is doing. It may be possible to create a machine that flirts with actual intelligence, but this is simply not it. There's not even room for doubt about this.

iameli 4y ago
I can't claim to understand everything my brain is doing, but accepting input and filtering it through a bunch of neuron chains to result in some kind of output sounds like "statistical analysis" to me. You seem convinced that ChatGPT will never have "actual intelligence" — care to make a prediction about something that LLMs will never accomplish? We know they can write code, write essays, generate artwork, and play chess. What's a task that requires "actual intelligence"? Parenting a child? Running for President? Making a steak sandwich?
- JohnFen 4y ago
  An LLM will never be able to produce something that cannot be statistically derived from its training data. So... creating a new language, maybe?
  - iameli 4y ago
    ChatGPT's attempt: https://gist.github.com/iameli/d9a5b715ec9baa5b11063888e054d36a https://gist.github.com/iameli/d9a5b715ec9baa5b11063888e054d... Are those two responses enough to say it's invented a language? Probably not. But it's already farther along than I would have gotten if you asked me to do such a thing. If I kept prompting it for hours, I bet it would start to contradict itself and lose track of the rules it had already established, but so would I. If I were actually inventing a language, I'd take months and keep extensive cross-referenced documentation on grammar and syntax and vocabulary. We don't _let_ ChatGPT do that, it has no mechanism for persisting its ideas like that. But like... neither would I if you took away my notebook and the parts of my brain that persist long-term memory. I guess my interpretation here - I see differences in _capabilities_, but not differences in actual _intelligence_.
  - neongreen 4y ago
    https://maximumeffort.substack.com/p/i-taught-chatgpt-to-invent-a-language https://maximumeffort.substack.com/p/i-taught-chatgpt-to-inv... My bet would be that in ten years GPT would get pretty good at it, but eh, I dunno.
  - jesterman 4y ago
    I see what you are going with here for it not being able to come up with original things. But I think that falls apart when you realize almost nothing humans do is original, everything is just building on other things.
    - JohnFen 4y ago
      The vast majority of what people do is this, yes. Just like how most of the time, we're all running on autopilot and our behaviors are pretty much just following scripts. "Meat robots". But it's not 100%.
School-Cotton 4y ago
> whatever you and I are doing, it's not simply statistical analysis. How do you know?
- JohnFen 4y ago
  Because I have seen genuine creativity and invention.
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]

lapinot 4y ago

LLM (and deep learning in general) are to AGI what bogosort is to sorting. For some reason beyond me people think it's very important to not try to understand anything about the structure of the problem you're trying to solve and just make a very general algorithm which will be suboptimal in just about every way except for the generality of it's code. Seeing the world as a trillion dimensional token soup is definitely quite general and at the same time very very weak in terms of expressivity.

iameli 4y ago
Sure, I think buy this interpretation. As long as we can agree that bogosort and quicksort are both still sorting algorithms. My brain definitely has some structures that are useful for me understanding the world that are more immediately useful than trillion dimensional token soup. But I'm also not convinced that it's impossible those structures could be successfully emulated by quadrillion dimensional token soup. And a lot of folks seem to be convinced that it's some kind of fundamental impossibility.
canjobear 4y ago
No other approach has worked as well for natural language, and not for lack of trying.

hathawsh 4y ago

It's because you and I also have mental resources that are structured very differently from ChatGPT. Even if some part of our brain might resemble a LLM, a LLM is a very poor representation of other parts of the brain. If you take ChatGPT too far, it falls into repetition and demonstrates that it obviously has no comprehension of the material it's remixing. Maybe some combination of a LLM and other mental machinery would result in AGI with real comprehension.

iameli 4y ago
I dunno - I hear what you're saying about how it seems to lose track of things over long conversations. But part of the mental machinery that it lacks is the ability to learn from those conversations and insert new ideas into its base model; every new conversation starts fresh. It has no long-term storage; it can't "keep notes" for itself, and we don't give it the ability to alter its fundamental state based on new input after we're done feeding it training data. I guess we're mostly agreeing, because those things are probably part of the "other mental machinery" we'd want to provide it, but I'd push back a little on "real comprehension". It kinda seems to me like the "comprehension engine" is working just fine, it's the structure we've built around that comprehension engine that's limiting it right now.

dinkumthinkum 4y ago

You’re comparing a LLM to yourself, really? I wonder how many people talking about this know much about computers. Your cognitive capability is much more complex than ChatGPT. I’m not sure why people are convinced that a Turing Machine can simulate it, let alone do it efficiently. Do you believe that the way you think is at all like ChatGPT. That you are both processing text is not evidence. How many watts of power did you utilize on your whole life up to this point? How much did it take to train ChatGPT and how much does it take to run it, even if you could run it on your home PC? How much heat does it generate. You may not like it but could live and solve intellectual problems for a week on only water and a handful of rice. Do you really think these two things, human brains and computers based on TMs are the same type of thing?

iameli 4y ago
Your argument is that my brain is more energy-efficient than ChatGPT, therefore... ChatGPT is incapable of "understanding"?
- dinkumthinkum 4y ago
  I’m not arguing that it is more energy efficient. I’m showing the obvious fact that there is a massive scale difference in the amount of energy and that one is capable of much more complex problem solving which seems to indicate that the two things are fundamentally different.
  - iameli 4y ago
    Okay. Same question I asked elsewhere in this thread; care to make a prediction about a problem that won't ever get solved by an LLM? We know they can write essays, create art, write code — what's something they'll never do?