6 ms·

It's interesting but also points out a flaw in a lot of people's thinking about this. Large language models have proven that AI doesn't need most aspects of per

by ilaksh 4y ago

It's interesting but also points out a flaw in a lot of people's thinking about this. Large language models have proven that AI doesn't need most aspects of personhood in order to be relatively general purpose. Humans and animals have: a stream of consciousness, deeply tied to the body and integration of numerous senses, a survival imperative, episodic memories, emotions for regulation, full autonomy, rapid learning, high adaptability. Large language models have none of those things. There is no reason to create these types of virtual hells for virtual people. Instead, build Star Trek-like computers (the ship's computer, not Data!) to order around. If you make virtual/artificial people, give them the same respect and rights as everyone.

13years 4y ago
Yes, it has shown that we might progress towards AGI without ever having anything that is sentient. It could be nearly imperceptible difference externally. Nonetheless, it brings forward a couple of other issues. We might never know if we have achieved sentience or just the resemblance of sentience. Furthermore, many of the concerns of AGI might still become an issue even if the machine does not technically "think".
safety1st 4y ago
Right. It's a great story but to me it's more of a commentary on modern Internet-era ethics than it is a prediction of the future. It's highly unlikely that we'll be scanning, uploading, and booting up brains in the cloud any time soon. This isn't the direction technology is going. If we could, though, the author's spot on that there would be millions of people who would do some horrific things to those brains, and there would be trillions of dollars involved. The whole attention economy is built around manipulating people's brains for profit and not really paying attention to how it harms them. The story is an allegory for that.
- TheUndead96 4y ago
  Out of curiosity, what would you say is hardest constraint on this (brain replication) happening? Do you think that it would be an limitation on imaging/scanning technology?
  - dullcrisp 4y ago
    What would you say is the biggest impediment towards building flying, talking unicorns with magic powers? Is it teaching the horses to talk?
    - Jeff_Brown 4y ago
      This doesn't seem fair but it made me laugh a lot.
  - deleted 4y ago
    [deleted]
  - asgraham 4y ago
    It's hard to say what the hardest constraint will be, at this point. Imaging and scanning are definitely hard obstacles; right now even computational power is a hard obstacle. There are 100 trillion synapses in the brain, none of which are simple. It's reasonable to assume you could need a KB (likely more tbh) to represent each one faithfully (for things like neurotransmitter binding rates on both ends, neurotransmitter concentrations, general morphology, secondary factors like reuptake), none of which is constant. That means 100 petabytes just to represent the brain. Then you have to simulate it, probably at submillisecond resolution. So you'd have 100 petabytes of actively changing values every millisecond or less. That's 100k petaflops, at a bare, bare, baaaare minimum, more like an exaflop. This ignores neurons since there are only like 86 billion of them, but they could be sufficiently more complex than synapses that they'd actually be the dominant factor. Who knows. This also ignores glia, since most people don't know anything about glia and most people assume that they don't do much with computation. Of course, when we have all the neurons represented perfectly, I'm sure we'll discover the glia need to be in there, too. There are about as many glia as neurons (3x more in the cortex, the part that makes you you, coloquially), and I've never seen any estimate of how many connections they have [1]. Bottom line: we almost certainly need exaflops to simulate a replicated brain, maybe zettaflops to be safe. Even with current exponential growth rates [2] (and assuming brain simulation can be simply parallelized (it can't)), that's like 45 years away. That sounds sorta soon, but I'm way more likely to be underestimating the scale of the problem than overestimating it, and that's how long until we can even begin trying. How long until we can meaningfully use those zettaflops is much, much longer. [1] I finished my PhD two months ago and my knowledge of glia is already outdated. We were taught glia outnumbered neurons 10-to-1: apparently this is no longer thought to be the case. https://en.wikipedia.org/wiki/Glia#Total_number https://en.wikipedia.org/wiki/Glia#Total_number [2] https://en.wikipedia.org/wiki/FLOPS#/media/File:Supercomputer_Power_(FLOPS),_OWID.svg https://en.wikipedia.org/wiki/FLOPS#/media/File:Supercompute...
scubakid 4y ago
I think many people who argue that LLMs could already be sentient are slow to grasp how fundamentally different it is that current models lack a consistent stream of perceptual inputs that result in real-time state changes. To me, it seems more like we've frozen the language processing portion of a brain, put it on a lab table, and now everyone gets to take turns poking it with a cattle prod.
- galaxytachyon 4y ago
  I talked about this sometime ago with another person. But at what point do we stop associating things with consciousness? Most people consider the brain is the seat of all that you are. But we also know how much the environment affect our "selves". Sunlight, food, temperature, other people, education, external knowledge, they all contribute quite significantly to your consciousness. Going the opposite way, religious people may disagree and say the soul is what actually you and nothing else matters. We can't even decide how much, and of what, would constitute a person. If like you said, the best AI right now is just a portion of the language processing part of our brain, it still can be sentient. Not that I think LLMs are anything close to people or AGI. But the fact is that we can't concretely and absolutely refute AI sentience based on our current knowledge. The technology deserves respect and deep thoughts instead of dismissing it as "glorified autocomplete". Nature needed billions of years to go from inert chemicals to sentience. We went from vacuum tubes to something resembling it in less than a century. Where can it go in the next century?
  - Jensson 4y ago
    A dead brain isn't conscious, most agree with that. But all the neural connections are still there, so you could inspect those and probably calculate what the human would respond to things, but I think the human is still dead even if you can now "talk" to him.
    - PebblesRox 4y ago
      Interesting to think about how we do use our mental models of people to predict how they would respond to things even after they're gone.

by ilaksh 4y ago

13years 4y ago

Yes, it has shown that we might progress towards AGI without ever having anything that is sentient. It could be nearly imperceptible difference externally. Nonetheless, it brings forward a couple of other issues. We might never know if we have achieved sentience or just the resemblance of sentience. Furthermore, many of the concerns of AGI might still become an issue even if the machine does not technically "think".

safety1st 4y ago

Right. It's a great story but to me it's more of a commentary on modern Internet-era ethics than it is a prediction of the future. It's highly unlikely that we'll be scanning, uploading, and booting up brains in the cloud any time soon. This isn't the direction technology is going. If we could, though, the author's spot on that there would be millions of people who would do some horrific things to those brains, and there would be trillions of dollars involved. The whole attention economy is built around manipulating people's brains for profit and not really paying attention to how it harms them. The story is an allegory for that.

TheUndead96 4y ago
Out of curiosity, what would you say is hardest constraint on this (brain replication) happening? Do you think that it would be an limitation on imaging/scanning technology?
- dullcrisp 4y ago
  What would you say is the biggest impediment towards building flying, talking unicorns with magic powers? Is it teaching the horses to talk?
  - Jeff_Brown 4y ago
    This doesn't seem fair but it made me laugh a lot.
- deleted 4y ago
  [deleted]
- asgraham 4y ago
  It's hard to say what the hardest constraint will be, at this point. Imaging and scanning are definitely hard obstacles; right now even computational power is a hard obstacle. There are 100 trillion synapses in the brain, none of which are simple. It's reasonable to assume you could need a KB (likely more tbh) to represent each one faithfully (for things like neurotransmitter binding rates on both ends, neurotransmitter concentrations, general morphology, secondary factors like reuptake), none of which is constant. That means 100 petabytes just to represent the brain. Then you have to simulate it, probably at submillisecond resolution. So you'd have 100 petabytes of actively changing values every millisecond or less. That's 100k petaflops, at a bare, bare, baaaare minimum, more like an exaflop. This ignores neurons since there are only like 86 billion of them, but they could be sufficiently more complex than synapses that they'd actually be the dominant factor. Who knows. This also ignores glia, since most people don't know anything about glia and most people assume that they don't do much with computation. Of course, when we have all the neurons represented perfectly, I'm sure we'll discover the glia need to be in there, too. There are about as many glia as neurons (3x more in the cortex, the part that makes you you, coloquially), and I've never seen any estimate of how many connections they have [1]. Bottom line: we almost certainly need exaflops to simulate a replicated brain, maybe zettaflops to be safe. Even with current exponential growth rates [2] (and assuming brain simulation can be simply parallelized (it can't)), that's like 45 years away. That sounds sorta soon, but I'm way more likely to be underestimating the scale of the problem than overestimating it, and that's how long until we can even begin trying. How long until we can meaningfully use those zettaflops is much, much longer. [1] I finished my PhD two months ago and my knowledge of glia is already outdated. We were taught glia outnumbered neurons 10-to-1: apparently this is no longer thought to be the case. https://en.wikipedia.org/wiki/Glia#Total_number https://en.wikipedia.org/wiki/Glia#Total_number [2] https://en.wikipedia.org/wiki/FLOPS#/media/File:Supercomputer_Power_(FLOPS),_OWID.svg https://en.wikipedia.org/wiki/FLOPS#/media/File:Supercompute...

scubakid 4y ago

I think many people who argue that LLMs could already be sentient are slow to grasp how fundamentally different it is that current models lack a consistent stream of perceptual inputs that result in real-time state changes. To me, it seems more like we've frozen the language processing portion of a brain, put it on a lab table, and now everyone gets to take turns poking it with a cattle prod.

galaxytachyon 4y ago
I talked about this sometime ago with another person. But at what point do we stop associating things with consciousness? Most people consider the brain is the seat of all that you are. But we also know how much the environment affect our "selves". Sunlight, food, temperature, other people, education, external knowledge, they all contribute quite significantly to your consciousness. Going the opposite way, religious people may disagree and say the soul is what actually you and nothing else matters. We can't even decide how much, and of what, would constitute a person. If like you said, the best AI right now is just a portion of the language processing part of our brain, it still can be sentient. Not that I think LLMs are anything close to people or AGI. But the fact is that we can't concretely and absolutely refute AI sentience based on our current knowledge. The technology deserves respect and deep thoughts instead of dismissing it as "glorified autocomplete". Nature needed billions of years to go from inert chemicals to sentience. We went from vacuum tubes to something resembling it in less than a century. Where can it go in the next century?
- Jensson 4y ago
  A dead brain isn't conscious, most agree with that. But all the neural connections are still there, so you could inspect those and probably calculate what the human would respond to things, but I think the human is still dead even if you can now "talk" to him.
  - PebblesRox 4y ago
    Interesting to think about how we do use our mental models of people to predict how they would respond to things even after they're gone.