4 ms·

Realizing that the model isn't having a cogent conversation with the user, that the output unravels into incoherence as you extend it enough and that the whole

Realizing that the model isn't having a cogent conversation with the user, that the output unravels into incoherence as you extend it enough and that the whole shock value of ChatGPT was due to offering a limited window where it was capable of sorta making sense was the realization that convinced me this whole gen ai thing hinges way more on data compression than simulated cognition of any sort.

golergka 3y ago
Why do you think that data compression and cognition are fundamentally different?
- namaria 3y ago
  The behavior of large language models compressing 20 years of internet and being incapable of showing any true understanding of the things described therein.
  - FredPret 3y ago
    A human also compresses many years of experience into one conversation. Does this reflect true understanding of the things described? Only the human doing the talking can know, and even that is on shaky ground. (if you don't understand something, will you always realize this? You have to know it a little bit to judge your own competence).
    - namaria 3y ago
      We compress data from many senses and can use that to interactively build inner models and filters for the data stream. The experience of psychedelics such as psylocibin and lsd can be summarized as disabling some of these filters. The deep dream trick google did a while back was a good illustration of hallucinations and also seen in some symptoms of schizophrenia. In my view that shows we are simulating some brain data processing functions. Results from the systems conducting these simulations are very far from the capabilities of humans but help shed light into how we work. Conflating these systems with the full cognitive range of human understanding is disingenuous at best.
      - FredPret 3y ago
        It clearly can't have human understanding without being a human. But that doesn't mean it can't have any understanding. You can represent every word in English in a vector database; this isn't how humans understand words, but it's not nothing and might be better in some ways. Fish swim, submarines sail.
      - mistermann 3y ago
        > The experience of psychedelics such as psylocibin and lsd can be summarized as disabling some of these filters. I was thinking last night about where (during the trip) the certainty aspect of the "realer than reality" sensation comes from... The theory I came up with is that the certainty comes from the delta between the two experiences, as opposed to (solely) the psychedelic experience itself. This assumes that one's read on normal reality at the time remains largely intact, which I believe is (often) the case. Further investigation is needed, I'm working from several years old memories.
  - CamperBob2 3y ago
    At some point, we'll have to define "true understanding." Now seems like a good time to start thinking about it.
    - namaria 3y ago
      If a person could talk cogently about something for a minute or two before descending into incoherent mumbling would you say they have true understanding of the things they said in that minute?
      - CamperBob2 3y ago
        If so, you'll have to credit ChatGPT4 with the ability to do just that.
        namaria 3y ago
        Funny how you ask a sharp question and suddenly people answer "ha check mate". Two replies and two fast claims of winning the argument in response but not one honest answer.
        CamperBob2 3y ago
        Did you have an actual point to make?
      - zo1 3y ago
        Sounds like every debate and argument I've ever had. You push and prod their argument for a few sentences back and forth and before you know it they start getting aggressive in their responses. Probably because they know they will soon devolve into a complete hallucinatory mess.
        namaria 3y ago
        Devolving into accusing me of aggression and implying I'm incapable of understanding the conversation for asking you a question sounds like you're the one avoiding it.
  - int_19h 3y ago
    There are many context in which it does show "true understanding", though, as evidenced by the ability to make new conclusions. Whether it has enough understanding is a separate question. Why should we treat the concept as a binary, when it's clearly not the case even for ourselves? These models we have now are ultimately still toy-sized. Why is it surprising that their "compression" of 20 years of Internet is so lossy?
- staticman2 3y ago
  Why would anyone think otherwise?
- Groxx 3y ago
  Ignoring where I personally draw my line in the sand: people claiming they're the same have literally only failed in demonstrating it, so it's not much of a scientific debate. It's a philosophy or dogma. It may be correct. Results are far from conclusive, or even supportive depending on interpretation.
- maximus-decimus 3y ago
  Because compressed data alone doesn't allow you to deal with new concept and theories? You think a data compression algorithm could have invented the atomic bomb?
- mplewis 3y ago
  Are you serious? Go outside.
altruios 3y ago
I read this, and I wonder: maybe cognition and data compression are closely related. We compress all our raw inputs into our brain into a somewhat wholistic experience - what is that other than compressing the data you experience world around you into a mental model of a query-able resolution?
- ChainOfFools 3y ago
  POETRY IS COMPRESSION William Goldman, the guy who wrote the screenplay for The Princess Bride among other things, claimed that this realization exposed the extraordinarily simple mechanism at work behind the most subjectively satisfying writing he had encountered of any form, though closest to the surface in the best poetry. further reminds me of another observation, not from Goldman but someone else I can't recall, to the effect that a poem is "a machine made of words."
  - fennecbutt 3y ago
    Interpretation is an extremely lossy mechanism, though
    - ChainOfFools 3y ago
      Very true, but it's an informed and curated loss. Necessarily so, because our couple kilograms lump of nerve tissue is completely unequal to the task of losslessly comprehending all of its own experiences, to say nothing of those of others, and infinitesimally so in comparison to the universe as a whole. We take points and interpolate a silhouette of reality from them. I am strongly on board with the notion that everything that we call knowledge or the human experience is all a lossy compression algorithm, a predatory consciousness imagining itself consuming the solid reality on which it presently floats an existence as a massless, insubstantial ghost.
  - spookybones 3y ago
    Where does he talk about this? I’m interested in reading it
fennecbutt 3y ago
Idt anybody reasonably involved ever claimed it was simulated cognition. It's just really good at predicting the next word. And tbf, human conversation that goes on too long can follow the same pattern, though models are disadvantaged by their context length. Imagine someone asked you to keep talking forever, but every 5 minutes they hit you in the head and you had no memory except from that point onwards. I'm sure I'd sound deranged, too.
- canadianfella 3y ago
  [dead]
- elicksaur 3y ago
  > i am a stochastic parrot, and so r u - Sam Altman, CEO of OpenAI https://twitter.com/sama/status/1599471830255177728 https://twitter.com/sama/status/1599471830255177728 But I’m sure he was joking. If he wasn’t, I’m sure he’s not actually reasonably involved. If he is, I’m sure he just didn’t mean that cognition was essentially a stochastic parrot. It’s pretty obvious what the people pushing LLM-style AI think about the human brain.
  - fieldcny 3y ago
    This is a wonderful comment, I’m sure he’s also not trying to raise $7T, or if he is it’s not US dollars…
- krapp 3y ago
  Human beings seem to be hard-wired to equate the appearance of coherent language with evidence of cognition. Even on Hacker News, where people should know better, a lot people seem to believe LLMs are literally sentient and self aware, not simply equivalent to but surpassing human capabilities in every dimension. I mean, I know a lot of that is simply the financial incentives of people whose job it is to push the Overton window of LLMs being recognized as legal beings equivalent to humans so that their training data is no longer subject to claims of copyright infringement (because it's simply "learning as a human mind would") but it also seems there's a deep seated human biological imperative being hacked here. The sociology behind the way people react to LLMs is fascinating.
ActorNightly 3y ago
Philosophically, compression and intelligence are the same thing. The decompression (which is the more important thing) involves a combination of original data of a certain size, paired with an algorithm, that can produce data of much bigger size and correct arrangement so it can be input into another system. Much in a way that there will probably be some algorithm along with a base set of training data that will result in something like reinforcement learning being run (which could include loops of simulating some systems and learning the outcome of experiments) that will eventually result in something that resembles a human intelligence, which is the vocal/visual dataset arranged correctly that we humans need to believe something that is intelligent. The question is how much you can compress something, which is measuring the intelligence of the algorithm. An hypothetical all powerful AGI == an algorithm that decompresses some initial data in to an accurate representation of reality in its sphere of influence including all the microscopic chaotic effects, into perpetuity, faster than reality happens (which means the decompressed data size for a time slice has more data than reality in that time slice) LLMs may seem like a good amount of compression, but in reality they aren't that extraordinary. GPT4 is probably to the tune of about ~1TB in size. If you look at Wikipedia compressed without media, its like 33TB -> 24 GB. So with about the same compression ratio, its not farfetched to see that GPT4 is pretty much human text compressed, with just an VERY efficient search algorithm built in. And, if you look at its architecture, you can see that is just a fancy map lookup with some form of interpolation.
- namaria 3y ago
  > accurate representation of reality in its sphere of influence including all the microscopic chaotic effects, into perpetuity, faster than reality happens This sounds like a newtonian universe. Reality has been proven to be indeterminate before observation, and assuming there is more then one observer in the universe, your equating data compression and full reality simulation to 'absolute intelligence' becomes untenable

golergka 3y ago

Why do you think that data compression and cognition are fundamentally different?

namaria 3y ago
The behavior of large language models compressing 20 years of internet and being incapable of showing any true understanding of the things described therein.
- FredPret 3y ago
  A human also compresses many years of experience into one conversation. Does this reflect true understanding of the things described? Only the human doing the talking can know, and even that is on shaky ground. (if you don't understand something, will you always realize this? You have to know it a little bit to judge your own competence).
  - namaria 3y ago
    We compress data from many senses and can use that to interactively build inner models and filters for the data stream. The experience of psychedelics such as psylocibin and lsd can be summarized as disabling some of these filters. The deep dream trick google did a while back was a good illustration of hallucinations and also seen in some symptoms of schizophrenia. In my view that shows we are simulating some brain data processing functions. Results from the systems conducting these simulations are very far from the capabilities of humans but help shed light into how we work. Conflating these systems with the full cognitive range of human understanding is disingenuous at best.
    - FredPret 3y ago
      It clearly can't have human understanding without being a human. But that doesn't mean it can't have any understanding. You can represent every word in English in a vector database; this isn't how humans understand words, but it's not nothing and might be better in some ways. Fish swim, submarines sail.
    - mistermann 3y ago
      > The experience of psychedelics such as psylocibin and lsd can be summarized as disabling some of these filters. I was thinking last night about where (during the trip) the certainty aspect of the "realer than reality" sensation comes from... The theory I came up with is that the certainty comes from the delta between the two experiences, as opposed to (solely) the psychedelic experience itself. This assumes that one's read on normal reality at the time remains largely intact, which I believe is (often) the case. Further investigation is needed, I'm working from several years old memories.
- CamperBob2 3y ago
  At some point, we'll have to define "true understanding." Now seems like a good time to start thinking about it.
  - namaria 3y ago
    If a person could talk cogently about something for a minute or two before descending into incoherent mumbling would you say they have true understanding of the things they said in that minute?
    - CamperBob2 3y ago
      If so, you'll have to credit ChatGPT4 with the ability to do just that.
      - namaria 3y ago
        Funny how you ask a sharp question and suddenly people answer "ha check mate". Two replies and two fast claims of winning the argument in response but not one honest answer.
        CamperBob2 3y ago
        Did you have an actual point to make?
    - zo1 3y ago
      Sounds like every debate and argument I've ever had. You push and prod their argument for a few sentences back and forth and before you know it they start getting aggressive in their responses. Probably because they know they will soon devolve into a complete hallucinatory mess.
      - namaria 3y ago
        Devolving into accusing me of aggression and implying I'm incapable of understanding the conversation for asking you a question sounds like you're the one avoiding it.
- int_19h 3y ago
  There are many context in which it does show "true understanding", though, as evidenced by the ability to make new conclusions. Whether it has enough understanding is a separate question. Why should we treat the concept as a binary, when it's clearly not the case even for ourselves? These models we have now are ultimately still toy-sized. Why is it surprising that their "compression" of 20 years of Internet is so lossy?
staticman2 3y ago
Why would anyone think otherwise?
Groxx 3y ago
Ignoring where I personally draw my line in the sand: people claiming they're the same have literally only failed in demonstrating it, so it's not much of a scientific debate. It's a philosophy or dogma. It may be correct. Results are far from conclusive, or even supportive depending on interpretation.
maximus-decimus 3y ago
Because compressed data alone doesn't allow you to deal with new concept and theories? You think a data compression algorithm could have invented the atomic bomb?
mplewis 3y ago
Are you serious? Go outside.

altruios 3y ago

I read this, and I wonder: maybe cognition and data compression are closely related. We compress all our raw inputs into our brain into a somewhat wholistic experience - what is that other than compressing the data you experience world around you into a mental model of a query-able resolution?

ChainOfFools 3y ago
POETRY IS COMPRESSION William Goldman, the guy who wrote the screenplay for The Princess Bride among other things, claimed that this realization exposed the extraordinarily simple mechanism at work behind the most subjectively satisfying writing he had encountered of any form, though closest to the surface in the best poetry. further reminds me of another observation, not from Goldman but someone else I can't recall, to the effect that a poem is "a machine made of words."
- fennecbutt 3y ago
  Interpretation is an extremely lossy mechanism, though
  - ChainOfFools 3y ago
    Very true, but it's an informed and curated loss. Necessarily so, because our couple kilograms lump of nerve tissue is completely unequal to the task of losslessly comprehending all of its own experiences, to say nothing of those of others, and infinitesimally so in comparison to the universe as a whole. We take points and interpolate a silhouette of reality from them. I am strongly on board with the notion that everything that we call knowledge or the human experience is all a lossy compression algorithm, a predatory consciousness imagining itself consuming the solid reality on which it presently floats an existence as a massless, insubstantial ghost.
- spookybones 3y ago
  Where does he talk about this? I’m interested in reading it

fennecbutt 3y ago

Idt anybody reasonably involved ever claimed it was simulated cognition. It's just really good at predicting the next word. And tbf, human conversation that goes on too long can follow the same pattern, though models are disadvantaged by their context length. Imagine someone asked you to keep talking forever, but every 5 minutes they hit you in the head and you had no memory except from that point onwards. I'm sure I'd sound deranged, too.

canadianfella 3y ago
[dead]
elicksaur 3y ago
> i am a stochastic parrot, and so r u - Sam Altman, CEO of OpenAI https://twitter.com/sama/status/1599471830255177728 https://twitter.com/sama/status/1599471830255177728 But I’m sure he was joking. If he wasn’t, I’m sure he’s not actually reasonably involved. If he is, I’m sure he just didn’t mean that cognition was essentially a stochastic parrot. It’s pretty obvious what the people pushing LLM-style AI think about the human brain.
- fieldcny 3y ago
  This is a wonderful comment, I’m sure he’s also not trying to raise $7T, or if he is it’s not US dollars…
krapp 3y ago
Human beings seem to be hard-wired to equate the appearance of coherent language with evidence of cognition. Even on Hacker News, where people should know better, a lot people seem to believe LLMs are literally sentient and self aware, not simply equivalent to but surpassing human capabilities in every dimension. I mean, I know a lot of that is simply the financial incentives of people whose job it is to push the Overton window of LLMs being recognized as legal beings equivalent to humans so that their training data is no longer subject to claims of copyright infringement (because it's simply "learning as a human mind would") but it also seems there's a deep seated human biological imperative being hacked here. The sociology behind the way people react to LLMs is fascinating.

ActorNightly 3y ago

Philosophically, compression and intelligence are the same thing. The decompression (which is the more important thing) involves a combination of original data of a certain size, paired with an algorithm, that can produce data of much bigger size and correct arrangement so it can be input into another system. Much in a way that there will probably be some algorithm along with a base set of training data that will result in something like reinforcement learning being run (which could include loops of simulating some systems and learning the outcome of experiments) that will eventually result in something that resembles a human intelligence, which is the vocal/visual dataset arranged correctly that we humans need to believe something that is intelligent. The question is how much you can compress something, which is measuring the intelligence of the algorithm. An hypothetical all powerful AGI == an algorithm that decompresses some initial data in to an accurate representation of reality in its sphere of influence including all the microscopic chaotic effects, into perpetuity, faster than reality happens (which means the decompressed data size for a time slice has more data than reality in that time slice) LLMs may seem like a good amount of compression, but in reality they aren't that extraordinary. GPT4 is probably to the tune of about ~1TB in size. If you look at Wikipedia compressed without media, its like 33TB -> 24 GB. So with about the same compression ratio, its not farfetched to see that GPT4 is pretty much human text compressed, with just an VERY efficient search algorithm built in. And, if you look at its architecture, you can see that is just a fancy map lookup with some form of interpolation.

namaria 3y ago
> accurate representation of reality in its sphere of influence including all the microscopic chaotic effects, into perpetuity, faster than reality happens This sounds like a newtonian universe. Reality has been proven to be indeterminate before observation, and assuming there is more then one observer in the universe, your equating data compression and full reality simulation to 'absolute intelligence' becomes untenable