4 ms·

I think the more interesting question is how long will people cling to the illusion that LLMs will lead us to AGI? Maintaining the illusion is important to kee

I think the more interesting question is how long will people cling to the illusion that LLMs will lead us to AGI? Maintaining the illusion is important to keep the money flowing in.

beepbooptheory 3y ago
While this is certainly true, I think we can't ignore the intense enthusiasm and faith of a large cohort of our peers (or, you know, HN commenters) who believe this to be The Way, and are not necessarily stakeholders in any meaningful sense. Just look at some of the responses even in this thread. It feels like some people just need this, and respond to balanced skepticism as Alyosha does to his brother Ivan. In part, whether conscious or not, people see the bright future of LLMs as a kind of redemption for the world so far wrought from a Silicon Valley ideology; its almost too on-the-nose the way chatgpt "fixes" internet search. But on a deeper level, consider how many hn posts we saw before chatgpt that were some variation of "I have reached a pinnacle of career accomplishment in the tech world, but I can't find meaning or value in my life." We don't seem to see those posts quite as much with all this AI stuff in the air. People seem to find some kind of existential value in the LLMs, one with an urgency that does not permit skepticism or critique. And, of course, in this thread alone, there is the constant refrain: "well, perhaps we are large language models ourselves after all..." This reflex to crude Skinnerism says a lot too: there are some that, I think, seek to be able to conquer even themselves; to reduce their inner life to python code and data, because it is something they can know and understand and thus have some kind of (sense) of control or insight about it. I don't want to be harsh saying this, people need something to believe in. I just think we can't discount how personal all this appears to be for a lot of regular, non-AI-CEO people. It is just extremely interesting, this culture and ideology being built around this. To me it rivals the LLMs themselves as a kind fascinating subject of inquiry.
- visarga 3y ago
  There is no magic in the brain. There is no magic in LLMs. There is just new experience we gain by interacting with the environment and society. And there is the trove of past experience encoded in our books. We got smart by collecting experience, in other words, from outside. The magic in the brain was not in the brain, but everywhere else. What is experience? We are in state S, and take action A, and observe feedback R. The environment is the teacher, giving us reward signals. We can only increase our knowledge incrementally, by trying our many bad ideas, and sometimes paying with our lives. But we still leave morsels of newly acquired experience for future generations. We are experience machines, both individually and socially. And intelligence is the distilled experience of the past, encoded in concepts, methods and knowledge. Intelligence is a collective process. None of us could reach our current level without language and society. Human language is in a way smarter than humans.
  - crowbahr 3y ago
    To say there's no magic in the brain drastically *minimizes the complexity of the brain. Your brain is several orders of magnitude more complex than even the largest LLM. GPT4 has 1 trillion parameters? Big deal. Your brain has 1 quadrillion synapses, constantly shifting. Beyond that the synapses are analog messages, not binary. Each synapse is approximately like 1000 transistors based on the granularity of messaging it can send and receive. It is temporally complex as well as structurally complex, well beyond anything we've ever made. I'm strongly in favor of AGI, for what it's worth, but LLMs aren't even scratching the surface. They're nowhere close to a human. They're a mediocre pastiche and it's equally possible that they're a dead end as it is that they'll ever be AGI.
    - visarga 3y ago
      That kind of explains why humans need to absorb less language to train. It still takes 25 years of focused study to become capable of pushing the frontier of knowledge a tiny bit.
    - gnaritas99 3y ago
      [dead]
    - tomduncalf 3y ago
      Re: synapses being analog messages, isn’t this sort of true of neural networks? In my understanding, the weights, biases and values flowing through the network are floating point numbers so I’d argue closer to analog than binary.
      - crowbahr 3y ago
        You know that's a good point that I didn't consider in my comment. LLMs use 16bit floats, which is approximately the same number of values. But then you get into the nuance of spiking vs non spiking neurons etc, which to my knowledge isn't emulated. The brain is extremely complex. I don't believe LLMs do anything similar to inhibitor neuronal function for example.
  - 3y ago
- mgaunard 3y ago
  As much as I personally believe that neural networks do bear a lot of resemblance to the human psyche, and that people are just sophisticated biological machines, I don't see how LLMs are capturing all of our thought processes. What I say is not just regurgitation of my past experiences; there is a logic to it.
ryanklee 3y ago
You say this is as if it's settled and obvious that it it's an illusion and it's only the delusional that believe the opposite. But if it were so settled and obvious there would be a clear line of reasoning to make that plain. And there is not. Instead, there is a very vibrant debate on the topic with tons of nuance and good faith (and bad) on each side, if we want to talk about sides. And, of course, one of the implications of this very real and significant inquiry that needs to be made and that requires real contributions from informed individuals, is that whenever anyone is dismissive or reductive regarding the unresolved difficulties, you can be sure they have absolutely no clue what they are talking about.
- mgaunard 3y ago
  There are tens of billions in the balance, how can you make sure the arguments are made in good faith? Money is self-reinforcing as well.
  - ryanklee 3y ago
    > how can you make sure the arguments are made in good faith Not sure why I would need to. I'm general, arguments can simply be understood. If they are good, good. If not, then fix them or move along. Good and bad faith are about people, not arguments.
gnaritas99 3y ago
[dead]

I think the more interesting question is how long will people cling to the illusion that LLMs will lead us to AGI? Maintaining the illusion is important to keep the money flowing in.

beepbooptheory 3y ago

While this is certainly true, I think we can't ignore the intense enthusiasm and faith of a large cohort of our peers (or, you know, HN commenters) who believe this to be The Way, and are not necessarily stakeholders in any meaningful sense. Just look at some of the responses even in this thread. It feels like some people just need this, and respond to balanced skepticism as Alyosha does to his brother Ivan. In part, whether conscious or not, people see the bright future of LLMs as a kind of redemption for the world so far wrought from a Silicon Valley ideology; its almost too on-the-nose the way chatgpt "fixes" internet search. But on a deeper level, consider how many hn posts we saw before chatgpt that were some variation of "I have reached a pinnacle of career accomplishment in the tech world, but I can't find meaning or value in my life." We don't seem to see those posts quite as much with all this AI stuff in the air. People seem to find some kind of existential value in the LLMs, one with an urgency that does not permit skepticism or critique. And, of course, in this thread alone, there is the constant refrain: "well, perhaps we are large language models ourselves after all..." This reflex to crude Skinnerism says a lot too: there are some that, I think, seek to be able to conquer even themselves; to reduce their inner life to python code and data, because it is something they can know and understand and thus have some kind of (sense) of control or insight about it. I don't want to be harsh saying this, people need something to believe in. I just think we can't discount how personal all this appears to be for a lot of regular, non-AI-CEO people. It is just extremely interesting, this culture and ideology being built around this. To me it rivals the LLMs themselves as a kind fascinating subject of inquiry.

visarga 3y ago
There is no magic in the brain. There is no magic in LLMs. There is just new experience we gain by interacting with the environment and society. And there is the trove of past experience encoded in our books. We got smart by collecting experience, in other words, from outside. The magic in the brain was not in the brain, but everywhere else. What is experience? We are in state S, and take action A, and observe feedback R. The environment is the teacher, giving us reward signals. We can only increase our knowledge incrementally, by trying our many bad ideas, and sometimes paying with our lives. But we still leave morsels of newly acquired experience for future generations. We are experience machines, both individually and socially. And intelligence is the distilled experience of the past, encoded in concepts, methods and knowledge. Intelligence is a collective process. None of us could reach our current level without language and society. Human language is in a way smarter than humans.
- crowbahr 3y ago
  To say there's no magic in the brain drastically *minimizes the complexity of the brain. Your brain is several orders of magnitude more complex than even the largest LLM. GPT4 has 1 trillion parameters? Big deal. Your brain has 1 quadrillion synapses, constantly shifting. Beyond that the synapses are analog messages, not binary. Each synapse is approximately like 1000 transistors based on the granularity of messaging it can send and receive. It is temporally complex as well as structurally complex, well beyond anything we've ever made. I'm strongly in favor of AGI, for what it's worth, but LLMs aren't even scratching the surface. They're nowhere close to a human. They're a mediocre pastiche and it's equally possible that they're a dead end as it is that they'll ever be AGI.
  - visarga 3y ago
    That kind of explains why humans need to absorb less language to train. It still takes 25 years of focused study to become capable of pushing the frontier of knowledge a tiny bit.
  - gnaritas99 3y ago
    [dead]
  - tomduncalf 3y ago
    Re: synapses being analog messages, isn’t this sort of true of neural networks? In my understanding, the weights, biases and values flowing through the network are floating point numbers so I’d argue closer to analog than binary.
    - crowbahr 3y ago
      You know that's a good point that I didn't consider in my comment. LLMs use 16bit floats, which is approximately the same number of values. But then you get into the nuance of spiking vs non spiking neurons etc, which to my knowledge isn't emulated. The brain is extremely complex. I don't believe LLMs do anything similar to inhibitor neuronal function for example.
- 3y ago
mgaunard 3y ago
As much as I personally believe that neural networks do bear a lot of resemblance to the human psyche, and that people are just sophisticated biological machines, I don't see how LLMs are capturing all of our thought processes. What I say is not just regurgitation of my past experiences; there is a logic to it.

ryanklee 3y ago

You say this is as if it's settled and obvious that it it's an illusion and it's only the delusional that believe the opposite. But if it were so settled and obvious there would be a clear line of reasoning to make that plain. And there is not. Instead, there is a very vibrant debate on the topic with tons of nuance and good faith (and bad) on each side, if we want to talk about sides. And, of course, one of the implications of this very real and significant inquiry that needs to be made and that requires real contributions from informed individuals, is that whenever anyone is dismissive or reductive regarding the unresolved difficulties, you can be sure they have absolutely no clue what they are talking about.

mgaunard 3y ago
There are tens of billions in the balance, how can you make sure the arguments are made in good faith? Money is self-reinforcing as well.
- ryanklee 3y ago
  > how can you make sure the arguments are made in good faith Not sure why I would need to. I'm general, arguments can simply be understood. If they are good, good. If not, then fix them or move along. Good and bad faith are about people, not arguments.

gnaritas99 3y ago

[dead]