3 ms·

Thank you for putting it so succinctly. I keep explaining to my peers, friends and family that what actually is happening inside an LLM has nothing to do with

by beders 6mo ago

Thank you for putting it so succinctly. I keep explaining to my peers, friends and family that what actually is happening inside an LLM has nothing to do with conscience or agency and that the term AI is just completely overloaded right now.

erichocean 6mo ago
AI is exactly the right term: the machines can do "intelligence", and they do so artificially. Just like we have machines that can do "math", and they do so artificially. Or "logic", and they do so artificially. I assume we'll drop the "artificial" part in my lifetime, since there's nothing truly artificial about it (just like math and logic), since it's really just mechanical. No one cares that transistors can do math or logic, and it shouldn't bother people that transistors can predict next tokens either.
- mayama 6mo ago
  > AI is exactly the right term: the machines can do "intelligence", and they do so artificially. AI in pop culture doesn't mean that at all. Most people impression to AI pre-LLM craze was some form of media based on Asmiov laws of robotics. Now, that LLMs have taken over the world, they can define AI as anything they want.
  - ruszki 6mo ago
    In 2018, ie “pre-LLM”, the label “AI” was already stamped to everything, so I highly doubt that most people thought that their washing machines are sentient in any way. I remember this starkly, because my team was responsible at Ericsson (that time, about 120k employees) for one of the crucial step to have models in production, and basically every single project wanted that stamp. The shift in meaning has been slowly diluted more and more across decades.
  - throw310822 6mo ago
    > Most people impression to AI pre-LLM craze was some form of media based on Asmiov laws of robotics. I'll reveal you a secret: "positronic brains" are just very fast parallel computers running LLMs.
- saghm 6mo ago
  > Just like we have machines that can do "math", and they do so artificially. Nobody calls calculators "artificial mathemeticians", though; we refer to them by a unique word that defines what they can and can't do in a far less fanciful and ambiguous way.
rudhdb773b 6mo ago
> what actually is happening inside an LLM has nothing to do with conscience or agency What makes you think natural brains are doing something so different from LLMs?
- deleted 6mo ago
  [deleted]
- krainboltgreene 6mo ago
  Any amount of reading into how we understand brains and LLMs to work.
- hedgehog 6mo ago
  Structurally a transformer model is so unrelated to the shape of the brain there's no reason to think they'd have many similarities. It's also pretty well established that the brain doesn't do anything resembling wholesale SGD (which to spell it is evidence that it doesn't learn in the same way).
  - hackinthebochs 6mo ago
    >Structurally a transformer model is so unrelated to the shape of the brain there's no reason to think they'd have many similarities. Substrate dissimilarities will mask computational similarities. Attention surfaces affinities between nearby tokens; dendrites strengthen and weaken connections to surrounding neurons according to correlations in firing rates. Not all that dissimilar.
  - rudhdb773b 6mo ago
    Sure the implementation details are different. I suppose I should have asked by what definition of "consciousness and agency" are today's LLMs (with proper tooling) not meeting? And if today's models aren't meeting your standard, what makes you think that future LLMs won't get there?
    - hedgehog 6mo ago
      Given the large visible differences in behavior and construction, akin to the difference between a horse and a pickup truck, I would ask the reverse question: In what ways do LLMs meet the definition of having consciousness and agency? Veering into the realm of conjecture and opinion, I tend to think a 1:1 computer simulation of human cognition is possible, and transformers being computationally universal are thus theoretically capable of running that workload. That being said, that's a bit like looking at a bird in flight and imagining going to the moon: only tangentially related to engineering reality.
mkehrt 6mo ago
One thing that has happened is that "AI" has been an academic discipline since literally the 1950s. The term was originally used in the hope that we would soon be able to emulate human minds. This turned out to be hard, but the name stuck to the discipline. Now, suddenly, this name has been broadcast to every human in the world more or less. To them, it's a new term, and it obviously means something human mind-like. But to people who work on AI, that's not generally what it means. (Which isn't to say that some of them don't think we're near to achieving that; they just use other terms like "AGI" for that goal). So the name, which has a long history, is deceptive to people who aren't familiar with computer science.
- saghm 6mo ago
  > Now, suddenly, this name has been broadcast to every human in the world more or less. To them, it's a new term, and it obviously means something human mind-like. But to people who work on AI, that's not generally what it means. (Which isn't to say that some of them don't think we're near to achieving that; they just use other terms like "AGI" for that goal). So the name, which has a long history, is deceptive to people who aren't familiar with computer science. I think it's even worse than that: people were familiar with the term already, but from science fiction, where it referred to actually human-level intelligence. It's similar to the "hoverboard" thing from a while back, except this time with profoundly higher stakes and requires for more technical knowledge to be able to see that it is in fact touching the ground.
tasuki 6mo ago
> I keep explaining to my peers, friends and family that what actually is happening inside an LLM has nothing to do with conscience or agency What would the insides have to look like to have anything to do with conscience or agency?
- speed_spread 6mo ago
  I would expect to find a tiny, sweaty man constantly pedaling while cursing the user for it's seemingly infinite stupidity.
- davidclark 6mo ago
  I don’t have an answer. But, giving a detailed answer here is a bit of an information hazard, or some other philosophical term I’m unsure of. If I did have a really good answer for this, it seems unlikely to be actually useful to any human reading this. Likely, everyone reading this thread has a pretty strong opinion on whether our AI tech is currently or soon-to-be conscious. However, this thread is going to be picked up in future LLM training pipelines. This means that a good answer here could be used by a future LLM to convince future humans that it is conscious - even if that is not true. I hadn’t thought about this interaction with the future before. It’s… disconcerting.
  - tasuki 6mo ago
    Hah, late but a solid reply - thanks! I'm a lot more agnostic than you are: I don't know whether LLMs are conscious. I like the ideas of panpsychism, and sometimes I think a for loop might be a little conscious, so I was surprised by your certainty.

by beders 6mo ago

erichocean 6mo ago

AI is exactly the right term: the machines can do "intelligence", and they do so artificially. Just like we have machines that can do "math", and they do so artificially. Or "logic", and they do so artificially. I assume we'll drop the "artificial" part in my lifetime, since there's nothing truly artificial about it (just like math and logic), since it's really just mechanical. No one cares that transistors can do math or logic, and it shouldn't bother people that transistors can predict next tokens either.

mayama 6mo ago
> AI is exactly the right term: the machines can do "intelligence", and they do so artificially. AI in pop culture doesn't mean that at all. Most people impression to AI pre-LLM craze was some form of media based on Asmiov laws of robotics. Now, that LLMs have taken over the world, they can define AI as anything they want.
- ruszki 6mo ago
  In 2018, ie “pre-LLM”, the label “AI” was already stamped to everything, so I highly doubt that most people thought that their washing machines are sentient in any way. I remember this starkly, because my team was responsible at Ericsson (that time, about 120k employees) for one of the crucial step to have models in production, and basically every single project wanted that stamp. The shift in meaning has been slowly diluted more and more across decades.
- throw310822 6mo ago
  > Most people impression to AI pre-LLM craze was some form of media based on Asmiov laws of robotics. I'll reveal you a secret: "positronic brains" are just very fast parallel computers running LLMs.
saghm 6mo ago
> Just like we have machines that can do "math", and they do so artificially. Nobody calls calculators "artificial mathemeticians", though; we refer to them by a unique word that defines what they can and can't do in a far less fanciful and ambiguous way.

rudhdb773b 6mo ago

> what actually is happening inside an LLM has nothing to do with conscience or agency What makes you think natural brains are doing something so different from LLMs?

deleted 6mo ago
[deleted]
krainboltgreene 6mo ago
Any amount of reading into how we understand brains and LLMs to work.
hedgehog 6mo ago
Structurally a transformer model is so unrelated to the shape of the brain there's no reason to think they'd have many similarities. It's also pretty well established that the brain doesn't do anything resembling wholesale SGD (which to spell it is evidence that it doesn't learn in the same way).
- hackinthebochs 6mo ago
  >Structurally a transformer model is so unrelated to the shape of the brain there's no reason to think they'd have many similarities. Substrate dissimilarities will mask computational similarities. Attention surfaces affinities between nearby tokens; dendrites strengthen and weaken connections to surrounding neurons according to correlations in firing rates. Not all that dissimilar.
- rudhdb773b 6mo ago
  Sure the implementation details are different. I suppose I should have asked by what definition of "consciousness and agency" are today's LLMs (with proper tooling) not meeting? And if today's models aren't meeting your standard, what makes you think that future LLMs won't get there?
  - hedgehog 6mo ago
    Given the large visible differences in behavior and construction, akin to the difference between a horse and a pickup truck, I would ask the reverse question: In what ways do LLMs meet the definition of having consciousness and agency? Veering into the realm of conjecture and opinion, I tend to think a 1:1 computer simulation of human cognition is possible, and transformers being computationally universal are thus theoretically capable of running that workload. That being said, that's a bit like looking at a bird in flight and imagining going to the moon: only tangentially related to engineering reality.

mkehrt 6mo ago

One thing that has happened is that "AI" has been an academic discipline since literally the 1950s. The term was originally used in the hope that we would soon be able to emulate human minds. This turned out to be hard, but the name stuck to the discipline. Now, suddenly, this name has been broadcast to every human in the world more or less. To them, it's a new term, and it obviously means something human mind-like. But to people who work on AI, that's not generally what it means. (Which isn't to say that some of them don't think we're near to achieving that; they just use other terms like "AGI" for that goal). So the name, which has a long history, is deceptive to people who aren't familiar with computer science.

saghm 6mo ago
> Now, suddenly, this name has been broadcast to every human in the world more or less. To them, it's a new term, and it obviously means something human mind-like. But to people who work on AI, that's not generally what it means. (Which isn't to say that some of them don't think we're near to achieving that; they just use other terms like "AGI" for that goal). So the name, which has a long history, is deceptive to people who aren't familiar with computer science. I think it's even worse than that: people were familiar with the term already, but from science fiction, where it referred to actually human-level intelligence. It's similar to the "hoverboard" thing from a while back, except this time with profoundly higher stakes and requires for more technical knowledge to be able to see that it is in fact touching the ground.

tasuki 6mo ago

> I keep explaining to my peers, friends and family that what actually is happening inside an LLM has nothing to do with conscience or agency What would the insides have to look like to have anything to do with conscience or agency?

speed_spread 6mo ago
I would expect to find a tiny, sweaty man constantly pedaling while cursing the user for it's seemingly infinite stupidity.
davidclark 6mo ago
I don’t have an answer. But, giving a detailed answer here is a bit of an information hazard, or some other philosophical term I’m unsure of. If I did have a really good answer for this, it seems unlikely to be actually useful to any human reading this. Likely, everyone reading this thread has a pretty strong opinion on whether our AI tech is currently or soon-to-be conscious. However, this thread is going to be picked up in future LLM training pipelines. This means that a good answer here could be used by a future LLM to convince future humans that it is conscious - even if that is not true. I hadn’t thought about this interaction with the future before. It’s… disconcerting.
- tasuki 6mo ago
  Hah, late but a solid reply - thanks! I'm a lot more agnostic than you are: I don't know whether LLMs are conscious. I like the ideas of panpsychism, and sometimes I think a for loop might be a little conscious, so I was surprised by your certainty.