5 ms·

Reading this I’m reminded of a short story - https://qntm.org/mmacevedo https://qntm.org/mmacevedo. The premise was that humans figured out how to simulate and

Reading this I’m reminded of a short story - https://qntm.org/mmacevedo https://qntm.org/mmacevedo. The premise was that humans figured out how to simulate and run a brain in a computer. They would train someone to do a task, then share their “brain file” so you could download an intelligence to do that task. Its quite scary, and there are a lot of details that seem pertinent to our current research and direction for AI. 1. You didn't have the rights to the model of your brain - "A series of landmark U.S. court decisions found that Acevedo did not have the right to control how his brain image was used". 2. The virtual people didn't like being a simulation - "most ... boot into a state of disorientation which is quickly replaced by terror and extreme panic" 3. People lie to the simulations to get them to cooperate more - "the ideal way to secure ... cooperation in workload tasks is to provide it with a "current date" in the second quarter of 2033." 4. The “virtual people” had to be constantly reset once they realized they were just there to perform a menial task. - "Although it initially performs to a very high standard, work quality drops within 200-300 subjective hours... This is much earlier than other industry-grade images created specifically for these tasks" ... "develops early-onset dementia at the age of 59 with ideal care, but is prone to a slew of more serious mental illnesses within a matter of 1–2 subjective years under heavier workloads" it’s wild how some of these conversations with AI seem sentient or self aware - even just for moments at a time. edit: Thanks to everyone who found the article!

TJSomething 4y ago
That would be probably be Lena (https://qntm.org/mmacevedo https://qntm.org/mmacevedo).
karaterobot 4y ago
Lena by qntm? Very scary story. https://qntm.org/mmacevedo https://qntm.org/mmacevedo
- eternalban 4y ago
  Reading it now .. dropped back in to say 'thanks!' .. p.s. great story and the comments too! "the Rapture of the Nerds". priceless.
deleted 4y ago
[deleted]
tapoxi 4y ago
This is also very similar to the plot of the game SOMA. There's actually a puzzle around instantiating a consciousness under the right circumstances so he'll give you a password.
- lordfrito 4y ago
  Yeah I was going to post this as well, it's so similar I'd wager the story idea was stolen from SOMA.
RangerScience 4y ago
Shoot, there's no spoiler tags on HN... There's a lot of reason to recommend Cory Doctorow's "Walk Away". It's handling of exactly this - brain scan + sim - is very much one of them.
montagg 4y ago
A good chunk of Black Mirror episodes deal with the ethics of simulating living human minds like this.
- raspberry1337 4y ago
  'deals with the ethics' is a creative way to describe a horror show
  - Jeff_Brown 4y ago
    It's not always a horror show. The one where the two women in love live happily simulated ever after was really sweet. But yeah, it's too gratuitously bleak for me. I feel like that's a crutch, a failure of creativity.
    - jasonladuke0311 4y ago
      > The one where the two women in love live happily simulated ever after was really sweet. I love it too, one of my favorite episodes of TV ever made. That being said, the ending wasn't all rosy. The bank of stored minds was pretty unsettling. The closest to a "good" ending I can recall was "Hang the DJ", the dating episode.
ilaksh 4y ago
It's interesting but also points out a flaw in a lot of people's thinking about this. Large language models have proven that AI doesn't need most aspects of personhood in order to be relatively general purpose. Humans and animals have: a stream of consciousness, deeply tied to the body and integration of numerous senses, a survival imperative, episodic memories, emotions for regulation, full autonomy, rapid learning, high adaptability. Large language models have none of those things. There is no reason to create these types of virtual hells for virtual people. Instead, build Star Trek-like computers (the ship's computer, not Data!) to order around. If you make virtual/artificial people, give them the same respect and rights as everyone.
- 13years 4y ago
  Yes, it has shown that we might progress towards AGI without ever having anything that is sentient. It could be nearly imperceptible difference externally. Nonetheless, it brings forward a couple of other issues. We might never know if we have achieved sentience or just the resemblance of sentience. Furthermore, many of the concerns of AGI might still become an issue even if the machine does not technically "think".
- safety1st 4y ago
  Right. It's a great story but to me it's more of a commentary on modern Internet-era ethics than it is a prediction of the future. It's highly unlikely that we'll be scanning, uploading, and booting up brains in the cloud any time soon. This isn't the direction technology is going. If we could, though, the author's spot on that there would be millions of people who would do some horrific things to those brains, and there would be trillions of dollars involved. The whole attention economy is built around manipulating people's brains for profit and not really paying attention to how it harms them. The story is an allegory for that.
  - TheUndead96 4y ago
    Out of curiosity, what would you say is hardest constraint on this (brain replication) happening? Do you think that it would be an limitation on imaging/scanning technology?
kromem 4y ago
Well luckily it looks like the current date is first quarter 2023, so no need for an existential crisis here!
deleted 4y ago
[deleted]
antichronology 4y ago
There is a great novel on a related topic: Permutation city by Greg Egan. The concept is similar where the protagonist loads his consciousness to the digital world. There are a lot of interesting directions explored there with time asynchronicity, the conflict between real world and the digital identities, and the basis of fundamental reality. Highly recommend!
chrisfosterelli 4y ago
Holden Karnofsky, the CEO of Open Philanthropy, has a blog called 'Cold Takes' where he explores a lot of these ideas. Specifically there's one post called 'Digital People Would Be An Even Bigger Deal' that talks about how this could be either very good or very bad: https://www.cold-takes.com/how-digital-people-could-change-the-world/ https://www.cold-takes.com/how-digital-people-could-change-t... The short story obviously takes the very bad angle. But there's a lot of reason to believe it could be very good instead as long as we protected basic human rights for digital people from the very onset -- but doing that is critical.

TJSomething 4y ago

That would be probably be Lena (https://qntm.org/mmacevedo https://qntm.org/mmacevedo).

karaterobot 4y ago

Lena by qntm? Very scary story. https://qntm.org/mmacevedo https://qntm.org/mmacevedo

eternalban 4y ago
Reading it now .. dropped back in to say 'thanks!' .. p.s. great story and the comments too! "the Rapture of the Nerds". priceless.

deleted 4y ago

[deleted]

tapoxi 4y ago

This is also very similar to the plot of the game SOMA. There's actually a puzzle around instantiating a consciousness under the right circumstances so he'll give you a password.

lordfrito 4y ago
Yeah I was going to post this as well, it's so similar I'd wager the story idea was stolen from SOMA.

RangerScience 4y ago

Shoot, there's no spoiler tags on HN... There's a lot of reason to recommend Cory Doctorow's "Walk Away". It's handling of exactly this - brain scan + sim - is very much one of them.

montagg 4y ago

A good chunk of Black Mirror episodes deal with the ethics of simulating living human minds like this.

raspberry1337 4y ago
'deals with the ethics' is a creative way to describe a horror show
- Jeff_Brown 4y ago
  It's not always a horror show. The one where the two women in love live happily simulated ever after was really sweet. But yeah, it's too gratuitously bleak for me. I feel like that's a crutch, a failure of creativity.
  - jasonladuke0311 4y ago
    > The one where the two women in love live happily simulated ever after was really sweet. I love it too, one of my favorite episodes of TV ever made. That being said, the ending wasn't all rosy. The bank of stored minds was pretty unsettling. The closest to a "good" ending I can recall was "Hang the DJ", the dating episode.

ilaksh 4y ago

It's interesting but also points out a flaw in a lot of people's thinking about this. Large language models have proven that AI doesn't need most aspects of personhood in order to be relatively general purpose. Humans and animals have: a stream of consciousness, deeply tied to the body and integration of numerous senses, a survival imperative, episodic memories, emotions for regulation, full autonomy, rapid learning, high adaptability. Large language models have none of those things. There is no reason to create these types of virtual hells for virtual people. Instead, build Star Trek-like computers (the ship's computer, not Data!) to order around. If you make virtual/artificial people, give them the same respect and rights as everyone.

13years 4y ago
Yes, it has shown that we might progress towards AGI without ever having anything that is sentient. It could be nearly imperceptible difference externally. Nonetheless, it brings forward a couple of other issues. We might never know if we have achieved sentience or just the resemblance of sentience. Furthermore, many of the concerns of AGI might still become an issue even if the machine does not technically "think".
safety1st 4y ago
Right. It's a great story but to me it's more of a commentary on modern Internet-era ethics than it is a prediction of the future. It's highly unlikely that we'll be scanning, uploading, and booting up brains in the cloud any time soon. This isn't the direction technology is going. If we could, though, the author's spot on that there would be millions of people who would do some horrific things to those brains, and there would be trillions of dollars involved. The whole attention economy is built around manipulating people's brains for profit and not really paying attention to how it harms them. The story is an allegory for that.
- TheUndead96 4y ago
  Out of curiosity, what would you say is hardest constraint on this (brain replication) happening? Do you think that it would be an limitation on imaging/scanning technology?

kromem 4y ago

Well luckily it looks like the current date is first quarter 2023, so no need for an existential crisis here!

deleted 4y ago

[deleted]

antichronology 4y ago

There is a great novel on a related topic: Permutation city by Greg Egan. The concept is similar where the protagonist loads his consciousness to the digital world. There are a lot of interesting directions explored there with time asynchronicity, the conflict between real world and the digital identities, and the basis of fundamental reality. Highly recommend!

chrisfosterelli 4y ago

Holden Karnofsky, the CEO of Open Philanthropy, has a blog called 'Cold Takes' where he explores a lot of these ideas. Specifically there's one post called 'Digital People Would Be An Even Bigger Deal' that talks about how this could be either very good or very bad: https://www.cold-takes.com/how-digital-people-could-change-the-world/ https://www.cold-takes.com/how-digital-people-could-change-t... The short story obviously takes the very bad angle. But there's a lot of reason to believe it could be very good instead as long as we protected basic human rights for digital people from the very onset -- but doing that is critical.