5 ms·

Clarity didn't work, trying mysterianism (2012)

82 points by dboon 2mo ago

avaer 2mo ago
One of the more compelling (and scary) stories about how AI takes over the world. If you liked this, also check our Pluribus.
- goatlover 2mo ago
  And there are people who argue in favor of the joined, because it's depicted as an entirely blissful state with no conflict (aside from the few remaining unjoined in the show). Despite the major downsides the first season has already shown.
mmooss 2mo ago
(2012) Also here: https://web.archive.org/web/20131230055512/http://squid314.livejournal.com/332946.html?thread=2852498#t2852498#t2852498 https://web.archive.org/web/20131230055512/http://squid314.l... Does anyone know the provenance of this story? One comment says, responding to someone who recommends publishing it, > The problem is that it's published so he can sell only second rights. EDIT: The story is tagged (at the top) "fiction, raikoth". Raikoth is the 'constructed civilization' of Scott Alexander, author of Slate Star Codex, et al. And though the post is at gwern.net, I see the author is "Scott".
- lalaithion 2mo ago
  What do you mean the provenance? It was published on livejournal in 2012, that’s the provenance.
- emmelaich 2mo ago
  Yep, Scott (squid314) used livejournal before SlateStarCodex, before AstralCodexTen.
beepbooptheory 2mo ago
This came up the other day, can someone help explain this to me? I am not quite sharp enough to understand it. Context: https://news.ycombinator.com/item?id=48911884 https://news.ycombinator.com/item?id=48911884
- fourthark 2mo ago
  Seems like a fairly direct parable against AI, suggesting that even if it's right it's better to think for yourself.
  - beepbooptheory 2mo ago
    You mean like LLM AI? Seems a little too early for that no? Either way, what is it right about? In your mind?
    - margalabargala 2mo ago
      It's not subtle, it's clearly saying "if you outsource your thinking to something else, your mind will wither". People complaining about this have a long and storied history, this is the same as Plato complaining philosophers were ruining their minds by writing things down to remember them.
      - beepbooptheory 2mo ago
        This is not what Plato thought, this is not even what Plato wrote that Socrates thought. You are referring to the Phaedrus. You should read it! Its much better than this story, trust me. But either way, people's minds do not wither in this story, it explicitly works out fine for them?
        Dylan16807 2mo ago
        > But either way, people's minds do not wither in this story, it explicitly works out fine for them? Huh? "when the corpses of the wearers were being prepared for burial, it was noted that their brains were curiously deformed: the neocortexes had wasted away, and the bulk of their mass was an abnormally hypertrophied mid- and lower-brain, especially the parts associated with reflexive action" You could infer their minds fading away even without that paragraph, but it's very explicit. Also you have another comment where you insist nobody has explained what the story is arguing when multiple people told you what the story is arguing, what the heck.
deleted 2mo ago
[deleted]
keeda 2mo ago
I suppose the earring never whispered anything about putting glue on pizza?
CamperBob2 2mo ago
I don't get it. First, how is it "better for you" to take the earring off, if "the wearer lives an abnormally successful life, usually ending out as a rich and much-beloved pillar of the community with a large and happy family?" Second, apparently some people do choose to take the earring off at some point, because the story refers to "the testimony of all living wearers." So it does not, in fact, appear to usurp, subvert or replace the wearer's free will. Not seeing the problem here.
- Nevermark 2mo ago
  How would you feel with a procedure in which your brain was scooped out like ice cream from a bowl, and replaced with circuits that would do a better job of your life for you? > So it does not, in fact, appear to usurp, subvert or replace the wearer's free will. Ok, the ice cream scooping happens slowly. At what point of atrophy, do you think it becomes easier to revert to what is left of it? You still have free will, in the strictest sense. But you are increasingly dependent, so have lost optionality that no amount of will can replace. Is your life story about what flows to you? Or about what challenges come to you, that you, yourself meet and overcome? EDIT: But you are also right - given discipline - the right thing to do is always leverage every piece of good advice we can get. From bot or earring. By always (1) first coming up with our own best plan, so we have a basis to recognize and appreciate the insights in better ideas (2) get the best possible feedback on that plan, the good and the bad, and then (3) with what we learned from that, attempt a better plan, and repeat until getting somewhere better that either initial plan, for all the looping. Then still learn from any failure or success. And do not go down the slippery slope of delegating unimportant questions, like choices of ice cream flavors! This is how I get the most from my AI advisors. To work and train myself faster and harder!
  - CamperBob2 2mo ago
    How would you feel with a procedure in which your brain was scooped out like ice cream from a bowl, and replaced with circuits that would do a better job of your life for you? One way to answer that might be to argue the counterfactual. How would I feel if I were born in the state described in the story -- little or no gray matter, but endowed with the promise of a successful, enjoyable life, valued and respected by those around me -- and someone offered me an earring whose advice would give rise to uncertainty, self-doubt, and a constant, nagging awareness of my own ignorance and lost potential? What incentive would I have to continue wearing that earring? For that version of the story to make sense, you'd almost have to make it a cursed object that couldn't be removed once equipped.
ShinyLeftPad 2mo ago
> The wearer lives an abnormally successful life as soon as most people wear it "abnormally" becomes "normally". logically the rare enlightened who refuse to wear it and instead keep their own [spoiler] become abnormal unsuccessful failures, not the other way around.
- raffael_de 2mo ago
  "It is no measure of health to be well adjusted to a profoundly sick society." Jiddu Krishnamurti
wdaher 2mo ago
I’ve also seen this called “The Whispering Earring”, and have seen it hosted here (this version slightly more readable) https://croissanthology.com/earring https://croissanthology.com/earring
ccapitalK 2mo ago
I wonder if there is a self consistent way to desire having your mind digitally uploaded into a computer, while detesting the idea of listening to the commands of the earring (and conditioning it to have all of my unconcious thoughts and desires). I guess I just discovered a bit of cognitive dissonance I didn't know I had.
- NooneAtAll3 2mo ago
  mind upload is closer to the version in the comments, with gem learning to emulate brain and then brain gets removed the difference is in targetted influence from outside aimed at changing you for its own goals and final form consider the "accelerated version" of this earpiece - where you just get whackked with dose of heroine (making you "perpetually happy") and then taking over your body
- andai 2mo ago
  I'll still be able to take my advice from Math.random()
Mikhail_Edoshin 2mo ago
This is a good companion to the yesterday's HN post of ten steps to happiness. I've read that Malachi Martin described what he called "a perfect possession". I do not know where as I haven't found this piece myself. But the idea was that typical cases of demonic possession are imperfect, the victim is struggling, and the struggle is visible to the outsiders. A perfect possession does not manifest itself.
citruscomputing 2mo ago
I've been thinking about this story a lot lately. When giving away your agency gives strictly better results, should you? And if you shouldn't, do you really think you're strong enough to resist it?
- goatlover 2mo ago
  Would you give away your agency to another human or human organization if they gave strictly better results? My guess is you value your own agency over your life more than that. It's your life to make mistakes and hopefully learn from (but that's up to you), not somebody else's to live for you.
andai 2mo ago
In the brain-chip case (or indeed already now, in the case of our rectangular appendages), some of the hisses and clicks come from mysterious sources beyond the earring.
eternauta3k 2mo ago
A philosopher or meditator should chime in here. Spending a modicum of time in meditation shows you that intentions just show up from nowhere, they're not "you". We might as well call the earring "your prefrontal cortex" (or whatever), and this would describe the current state of things.
rini17 2mo ago
At first glance it can work only by completely disabling the meaning-seeker part of the wearer's personality. Ennui and existential/absurd feelings can't be resolved by indulgence. Would it make Sisyphus happy without bypassing the implied will of gods that he shall be forever fully conscious in order to suffer? Someone pls give the earring to poor dude.
codebastard 2mo ago
[dead]

82 points by dboon 2mo ago

avaer 2mo ago

One of the more compelling (and scary) stories about how AI takes over the world. If you liked this, also check our Pluribus.

goatlover 2mo ago
And there are people who argue in favor of the joined, because it's depicted as an entirely blissful state with no conflict (aside from the few remaining unjoined in the show). Despite the major downsides the first season has already shown.

mmooss 2mo ago

(2012) Also here: https://web.archive.org/web/20131230055512/http://squid314.livejournal.com/332946.html?thread=2852498#t2852498#t2852498 https://web.archive.org/web/20131230055512/http://squid314.l... Does anyone know the provenance of this story? One comment says, responding to someone who recommends publishing it, > The problem is that it's published so he can sell only second rights. EDIT: The story is tagged (at the top) "fiction, raikoth". Raikoth is the 'constructed civilization' of Scott Alexander, author of Slate Star Codex, et al. And though the post is at gwern.net, I see the author is "Scott".

lalaithion 2mo ago
What do you mean the provenance? It was published on livejournal in 2012, that’s the provenance.
emmelaich 2mo ago
Yep, Scott (squid314) used livejournal before SlateStarCodex, before AstralCodexTen.

beepbooptheory 2mo ago

This came up the other day, can someone help explain this to me? I am not quite sharp enough to understand it. Context: https://news.ycombinator.com/item?id=48911884 https://news.ycombinator.com/item?id=48911884

fourthark 2mo ago
Seems like a fairly direct parable against AI, suggesting that even if it's right it's better to think for yourself.
- beepbooptheory 2mo ago
  You mean like LLM AI? Seems a little too early for that no? Either way, what is it right about? In your mind?
  - margalabargala 2mo ago
    It's not subtle, it's clearly saying "if you outsource your thinking to something else, your mind will wither". People complaining about this have a long and storied history, this is the same as Plato complaining philosophers were ruining their minds by writing things down to remember them.
    - beepbooptheory 2mo ago
      This is not what Plato thought, this is not even what Plato wrote that Socrates thought. You are referring to the Phaedrus. You should read it! Its much better than this story, trust me. But either way, people's minds do not wither in this story, it explicitly works out fine for them?
      - Dylan16807 2mo ago
        > But either way, people's minds do not wither in this story, it explicitly works out fine for them? Huh? "when the corpses of the wearers were being prepared for burial, it was noted that their brains were curiously deformed: the neocortexes had wasted away, and the bulk of their mass was an abnormally hypertrophied mid- and lower-brain, especially the parts associated with reflexive action" You could infer their minds fading away even without that paragraph, but it's very explicit. Also you have another comment where you insist nobody has explained what the story is arguing when multiple people told you what the story is arguing, what the heck.

deleted 2mo ago

[deleted]

keeda 2mo ago

I suppose the earring never whispered anything about putting glue on pizza?

CamperBob2 2mo ago

I don't get it. First, how is it "better for you" to take the earring off, if "the wearer lives an abnormally successful life, usually ending out as a rich and much-beloved pillar of the community with a large and happy family?" Second, apparently some people do choose to take the earring off at some point, because the story refers to "the testimony of all living wearers." So it does not, in fact, appear to usurp, subvert or replace the wearer's free will. Not seeing the problem here.

Nevermark 2mo ago
How would you feel with a procedure in which your brain was scooped out like ice cream from a bowl, and replaced with circuits that would do a better job of your life for you? > So it does not, in fact, appear to usurp, subvert or replace the wearer's free will. Ok, the ice cream scooping happens slowly. At what point of atrophy, do you think it becomes easier to revert to what is left of it? You still have free will, in the strictest sense. But you are increasingly dependent, so have lost optionality that no amount of will can replace. Is your life story about what flows to you? Or about what challenges come to you, that you, yourself meet and overcome? EDIT: But you are also right - given discipline - the right thing to do is always leverage every piece of good advice we can get. From bot or earring. By always (1) first coming up with our own best plan, so we have a basis to recognize and appreciate the insights in better ideas (2) get the best possible feedback on that plan, the good and the bad, and then (3) with what we learned from that, attempt a better plan, and repeat until getting somewhere better that either initial plan, for all the looping. Then still learn from any failure or success. And do not go down the slippery slope of delegating unimportant questions, like choices of ice cream flavors! This is how I get the most from my AI advisors. To work and train myself faster and harder!
- CamperBob2 2mo ago
  How would you feel with a procedure in which your brain was scooped out like ice cream from a bowl, and replaced with circuits that would do a better job of your life for you? One way to answer that might be to argue the counterfactual. How would I feel if I were born in the state described in the story -- little or no gray matter, but endowed with the promise of a successful, enjoyable life, valued and respected by those around me -- and someone offered me an earring whose advice would give rise to uncertainty, self-doubt, and a constant, nagging awareness of my own ignorance and lost potential? What incentive would I have to continue wearing that earring? For that version of the story to make sense, you'd almost have to make it a cursed object that couldn't be removed once equipped.

ShinyLeftPad 2mo ago

> The wearer lives an abnormally successful life as soon as most people wear it "abnormally" becomes "normally". logically the rare enlightened who refuse to wear it and instead keep their own [spoiler] become abnormal unsuccessful failures, not the other way around.

raffael_de 2mo ago
"It is no measure of health to be well adjusted to a profoundly sick society." Jiddu Krishnamurti

wdaher 2mo ago

I’ve also seen this called “The Whispering Earring”, and have seen it hosted here (this version slightly more readable) https://croissanthology.com/earring https://croissanthology.com/earring

ccapitalK 2mo ago

I wonder if there is a self consistent way to desire having your mind digitally uploaded into a computer, while detesting the idea of listening to the commands of the earring (and conditioning it to have all of my unconcious thoughts and desires). I guess I just discovered a bit of cognitive dissonance I didn't know I had.

NooneAtAll3 2mo ago
mind upload is closer to the version in the comments, with gem learning to emulate brain and then brain gets removed the difference is in targetted influence from outside aimed at changing you for its own goals and final form consider the "accelerated version" of this earpiece - where you just get whackked with dose of heroine (making you "perpetually happy") and then taking over your body
andai 2mo ago
I'll still be able to take my advice from Math.random()

Mikhail_Edoshin 2mo ago

This is a good companion to the yesterday's HN post of ten steps to happiness. I've read that Malachi Martin described what he called "a perfect possession". I do not know where as I haven't found this piece myself. But the idea was that typical cases of demonic possession are imperfect, the victim is struggling, and the struggle is visible to the outsiders. A perfect possession does not manifest itself.

citruscomputing 2mo ago

I've been thinking about this story a lot lately. When giving away your agency gives strictly better results, should you? And if you shouldn't, do you really think you're strong enough to resist it?

goatlover 2mo ago
Would you give away your agency to another human or human organization if they gave strictly better results? My guess is you value your own agency over your life more than that. It's your life to make mistakes and hopefully learn from (but that's up to you), not somebody else's to live for you.

andai 2mo ago

In the brain-chip case (or indeed already now, in the case of our rectangular appendages), some of the hisses and clicks come from mysterious sources beyond the earring.

eternauta3k 2mo ago

A philosopher or meditator should chime in here. Spending a modicum of time in meditation shows you that intentions just show up from nowhere, they're not "you". We might as well call the earring "your prefrontal cortex" (or whatever), and this would describe the current state of things.

rini17 2mo ago

At first glance it can work only by completely disabling the meaning-seeker part of the wearer's personality. Ennui and existential/absurd feelings can't be resolved by indulgence. Would it make Sisyphus happy without bypassing the implied will of gods that he shall be forever fully conscious in order to suffer? Someone pls give the earring to poor dude.

codebastard 2mo ago

[dead]