15 ms·

Crimes against transhumanity

167 points by zdw 4y ago

Symmetry 4y ago
I don't expect mind emulations to ever be an economical alternative to AIs. But if they are Robin Hanson wrote a book[1] trying to think through what would end up happening using as much boring social science and engineering as he could manage. I'm not convinced by everything but it's probably a good starting point for thinking through the issues. [1]https://en.wikipedia.org/wiki/The_Age_of_Em https://en.wikipedia.org/wiki/The_Age_of_Em EDIT: A better review than just the Wikiepdia page: https://slatestarcodex.com/2016/05/28/book-review-age-of-em/ https://slatestarcodex.com/2016/05/28/book-review-age-of-em/
- wging 4y ago
  Of that review, Hanson wrote: "If I ever have an executioner, I want him to be Scott Alexander." https://www.overcomingbias.com/2016/05/alexander-on-age-of-em.html https://www.overcomingbias.com/2016/05/alexander-on-age-of-e...
subjectsigma 4y ago
Playing devil's advocate for a second. I saw an article on HN a while ago written by a wealthy Asian male who was naturally very curious and interested in math. The article was about how he referred to himself as 'privileged', not only because he was born with wealth and therefore access to comfort and resources, but because his innate desire to learn made him much more capable and marketable in a modern workforce. He recognized that some people are not cognitively capable of learning higher mathematics, and part of that was because they just simply didn't have the drive, attention span, etc. to study hard enough to get into the highest levels, whereas he did. He is, by his definition, 'better', because of the roll of some cosmic dice. If we are to consider this definition of privilege to be acceptable, then simulated brains are pretty much the most privileged a human can be. They can re-clock their brain to think faster than humans. They can modify their own software to add in virtues and erase vices. And most importantly of all they are completely free from physical needs. If for some reason they desire scarcity or biological functions, that can all be simulated. Much like fears about genetic engineering of super people, the governments of the real world would have to move extremely quickly to protect the underprivileged class, e.g. the biologicals, otherwise they would be completely driven out of the highest paying intellectual and creative jobs, creating an economic lower class with no chance of ever out-competing their simulated counterparts.
- AnimalMuppet 4y ago
  > If we are to consider this definition of privilege to be acceptable, then simulated brains are pretty much the most privileged a human can be. Not so fast. A simulated brain may simulate a brain that lacks the "innate desire to learn". Then you'd have a fast brain that is still "underprivileged", by this definition.
  - xwdv 4y ago
    I think such brains would be killed off quickly and another one created until it is within the acceptable parameters.
    - test6554 4y ago
      I mean think of the horrors you could force upon a simulated brain without any consequences. Simply make a copy, do whatever torture you want to that copy until they are broken in every possible way, then delete the copy and the original would never know you have all their secrets. Heck, think if someone that was still in their physical body had an uploaded copy online. Nefarious actors could run simulations against a copy of that person's mind to find out which marketing techniques and sales pitches were most effective or which court arguments would sway them if they were ever on a jury. That doesn't sound very privileged to me. Sounds like some triggers/traps would need to be in place to know whether someone has accessed your mind against your will.
      - wging 4y ago
        There's so much fiction that deals with this beyond what Stross mentions, though he had finite space. On the topic of torturing someone in simulation (at high speed), see Altered Carbon.
      - xwdv 4y ago
        Hmm, would you agree of having copies of your brain tortured repeatedly at high frequency for marketing split testing purposes if it meant you could use a very valuable service for free?
gmuslera 4y ago
The referred short story, Lena (https://qntm.org/mmacevedo https://qntm.org/mmacevedo) is amazing. It did a better work than did severel Black Mirror episodes (specially a section in the Christmas one) warning us about what would happen if we ever manage to digitize personalities. In the end, things are always rigged towards those who have enough resources to do this kind of things and want to make more. If present crimes against humanism get a free ride, what we should expect about crimes against transhumanism?
- LeifCarrotson 4y ago
  Present crimes against humanism do not get a free ride! The author explicitly said so in this recent post [1] > ...uploading isn't real (now), so the things "Lena" has to say about uploading are academic (now). > The reason "Lena" is a concerning story isn't that one day we may be able to upload one another and when that happens we will do terrible things to those uploads. ... This is about appetites which, as we are all uncomfortably aware, already exist within human nature. > Oh boy, what if there was a maligned sector of human society whose members were for some reason considered less than human? What if they were less visible than most people, or invisible, and were exploited and abused, and had little ability to exercise their rights or even make their plight known? > That's real! That actually happens! You can name four groups of people matching that description without even thinking. We don't need to add some manufactured debate about fictitious, magical uploads to these real scenarios. They are already terrible! ... > "Lena" is a true story. You knew it was when you read it. > So, what do we do about this? In reality? ... > I've got no clue. It turns out that causing problems is a lot easier than organising against them, and I am just a science fiction writer. > Good luck, everyone. [1] https://qntm.org/uploading https://qntm.org/uploading
  - gmuslera 4y ago
    Would you consider funding a denialism campaing to stop action against climate change for decades a crime against humanity? Has been done anything against them? Murder is worse than abuse.
- mah4k4l 4y ago
shagie 4y ago
> Which leads me to ask: in a transhumanist society—go read Accelerando, or Glasshouse, or The Rapture of the Nerds—what currently recognized crimes need to be re-evaluated because their social impact has changed? And what strange new crimes might universally be recognized by a society with, for example, mind uploading, strong AI, or near-immortality? The link to Accelerando there is to Amazon (which is fine, and I do have physical and digital versions of the book and feel that it is good and useful to support authors by purchasing the stories they write)... but you can also get it for free, in its entirety, in a number of different formats, from the author's website: https://www.antipope.org/charlie/blog-static/fiction/accelerando/accelerando-intro.html https://www.antipope.org/charlie/blog-static/fiction/acceler... > This free ebook edition is made available by kind consent of my publishers, Ace and Orbit, under a Creative Commons license with certain restrictions attached. In particular, you may not create derivative works or use the work for commercial gain. (late edit - I personally feel that the sequence of books and stories: BLIT stories by David Langford, Accelerando (references a Langford fractal), Glasshouse (hints of {spoilers} in a transhuman environment, feels sequelish to Accelerando), Implied Spaces by Walter Jon Williams (the Strossian singularity is averted, and explores {spoilers} more) ---- There's a kind of continuity in the refutation of the reference that each makes to the other that I find interesting)
slfnflctd 4y ago
My parents are religious fundamentalists, and I've noticed there is increasing FUD being drummed up in their circles about anything to do with AI or significantly altering "what it means to be 'human'". The former is described as a front for evil spirits/demons, and the latter as the penultimate step before apocalyptic events, i.e. a precursor to the worst kinds of evil (in some circles, the 'mark of The Beast'). [For the record, I'm agnostic and feel the author raises some excellent points in any case.] There will be extreme resistance to these technologies from many kinds of religious people who would be willing to die and/or kill to prevent what they see as a takeover by the most malevolent of invisible forces. I may not agree with their reasoning, but some of their concerns could be worth reframing and considering in a different light. Personally, I strongly suspect we will have to fundamentally change as a species to get through at least one of the Great Filters looming before us. However, it's important to remember that the road is fraught with many perils and several possible paths which could lead to unimaginable suffering. It really seems to me that we're going to have to get lucky on multiple dice rolls here in the long run.
- api 4y ago
  > some of their concerns could be worth reframing and considering in a different light. There are some legitimate concerns about AI that sound almost like something out of the Book of Revelations, but are entirely rational. The biggest one is the idea of AI being used to implement "automated con artistry at scale." Imagine assigning every living human being a virtual AI powered con artist to tail them around and try to convince them of whatever is ordered by the highest bidder. This AI is powered by mass surveillance and big data and is able to know the "mark" intimately and work on them 24/7 both directly and via their social network. Now throw in deep fakes, attention maximizing "compulsion loops" and other adversarial models of human cognition, etc. It's basically the hydrogen bomb of propaganda, a doomsday machine for the mind. It would be like creating Satan, except this one would be entirely amoral and mercenary. At the very least this would be the end of democracy in any form. I could see this ushering in the permanent eternal victory of totalitarianism since I can't imagine the masses being able to organize any resistance when constantly bombarded with propaganda from their demons. The new feudal aristocracy would be the ones running the demons. This is one of the darkest plausible visions of the future I can think of at the moment, far worse than anything related to climate change or similar problems. If we manage to avoid these kinds of scenarios but still drown Miami I'll say we didn't do too badly. Transhumanism comes with some analogous concerns around the potential for enhanced humans to enslave the rest of humanity through superior cognitive capacities and the incredible accumulation of wealth. I can imagine a scenario where a few wealthy people gain access to technologies for life extension and cognitive enhancement and then "run away," effectively becoming a new species and exterminating or enslaving "legacy humans." This isn't mutually exclusive to the AI hellscape I describe above, since that would be an ideal mechanism to enslave the rest. Ultimately I think these technologies are all neutral. The problem is that we are not. I don't fear AI. I fear what humans will do with AI.
sk8terboi 4y ago
betwixthewires 4y ago
Nitpick, the author drops closing parentheses too often. Proofread and edit. I don't believe mind uploading is possible, so the particular examples cited here don't apply, but I can see how social principles can be upended by a revolutionary change in how we live, particularly one that tweaks the definition of what we are. Still, they're principles, they're not going away, just at best being reframed.
jmyeet 4y ago
I don't believe "mind uploading" will ever go anywhere. There's a real Ship of Theseus philosophical problem here and it's never provable so I expect many people would just think you've created a copy that superficially acts like the original while destroying the original. Technologically speaking, I'm not sure it's even possible to replicate the complex state of an entire nervous system. Maybe you could do this incrementally (ie by replacing parts of the brain with machines that can be replicated) but we're awfully sentimental about our own meat bags.
- twblalock 4y ago
  > so I expect many people would just think you've created a copy that superficially acts like the original while destroying the original. Prove otherwise. Star Trek transporters have the same issue.
  - SketchySeaBeast 4y ago
    That episode of TNG where Barclay sees and even interacts with weird slug things that turn out to be people while transporting seems to indicate that there's a continuity of awareness, which really raises even more questions and should probably be best ignored.
- imdsm 4y ago
  That's my thought on Stargates. Wouldn't your consciousness just stop as you were ripped apart? And a clone of you would then continue but it wouldn't be your stream. The more I think about this, about what my stream of consciousness is, my "session" if you will, the weirder it all gets.
  - bratbag 4y ago
    Especially when you consider sleeping. Do we die every night, with a copy continuing on tomorrow?
    - NateEag 4y ago
fennecfoxen 4y ago
The Federov link towards the end of this piece remindeded me of some older science fiction on the topic, specifically, C. S. Lewis, That Hideous Strength (published 1945, just before the atomic bomb got the genre's attention.) In this work, the central conspiracy has just (haltingly) achieved the artificial resurrection of a human brain, and this capability is the lynchpin of their scheme to take over the world. In the excerpt below, our protagonist Mark finally learns of the advance from the scientist Filostrato, while the slightly deranged preacher Straik theologizes (he might have been Federov-inspired.) It's all very much in line with the incredible possibilities of consciousness-abuse that is covered here. -- "It is the beginning of Man Immortal and Man Ubiquitous," said Straik. "Man on the throne of the universe. It is what all the prophecies really meant." "At first, of course," said Filostrato, "the power will be confined to a number--a small number--of individual men. Those who are selected for eternal life." "And you mean," said Mark, "it will then be extended to all men?" "No," said Filostrato. "I mean it will then be reduced to one man. You are not a fool, are you, my young friend? All that talk about the power of Man over Nature--Man in the abstract--is only for the canaglia. You know as well as I do that Man's power over Nature means the power of some men over other men with Nature as the instrument. There is no such thing as Man--it is a word. There are only men. No! It is not Man who will be omnipotent, it is some one man, some immortal man. Alcasan, our Head, is the first sketch of it. The completed product may be someone else. It may be you. It may be me." "A king cometh," said Straik, "who shall rule the universe with righteousness and the heavens with judgement. You thought all that was mythology, no doubt. You thought because fables had clustered about the phrase 'Son of Man' that Man would never really have a son who will wield all power. But he will." "I don't understand, I don't understand," said Mark. "But it is very easy," said Filostrato. "We have found how to make a dead man live. He was a wise man even in his natural life. He live now forever: he get wiser. Later, we make them live better--for at present, one must concede, this second life is probably not very agreeable to him who has it. You see? Later we make it pleasant for some--perhaps not so pleasant for others. For we can make the dead live whether they wish it or not. He who shall be finally king of the universe can give this life to whom he pleases. They cannot refuse the little present." "And so," said Straik, "the lessons you learned at your mother's knee return. God will have power to give eternal reward and eternal punishment."
- 4y ago
upupandup 4y ago
[flagged]
- MerelyMortal 4y ago
  Thank you for sharing that clip. Has anyone watched the series, Years and Years? It seems interesting.
- kleer001 4y ago
  https://en.wikipedia.org/wiki/Transhumanism https://en.wikipedia.org/wiki/Transhumanism "Julian Huxley was a biologist who popularised the term transhumanism in an influential 1957 essay." And IMHO it's a modern mutation of Gnosticism, which is older than dirt. https://en.wikipedia.org/wiki/Gnosticism https://en.wikipedia.org/wiki/Gnosticism Some people will always want to transcend their mortal birth.
- pessimizer 4y ago
  The West is jealous of theocracies, so it's retreating into bizarre fantasies that it's trying to harden into new secular religions that can be enforced by technology or law.
  - detaro 4y ago
    What does Transhumanism try to "enforce"?
- trasz 4y ago
  There are no TikTok trends that would be more stupid than what previous generations believed in.
deleted 4y ago
[deleted]
goodside 4y ago
For anyone wondering why the title of the short story is “Lena”, see: https://en.m.wikipedia.org/wiki/Lenna https://en.m.wikipedia.org/wiki/Lenna
motohagiography 4y ago
> Our intuitions about crimes against people (and humanity) are based on a set of assumptions about the parameters of personhood that are going to be completely destroyed if mind uploading turns out to be possible. There are human 'proof of work' activities that unless you can literally transplant the muscle memory, there is no way to replicate them without putting in the time, and I am saying without that effect of muscle memory, anything you transmit is just an artifact of language and narrative, and therefore not material. The advantages of augmented transhuman life aren't material, they're only symbolic in narrative. Imo, the example of a parliament trying and executing a formerly divinely appointed King was just the effect of the victory of a narrative and linguistic definition of self over a physical/spiritual orientation of self. It was just the rejection of theism. It wasn't world ending to be able to depose a king, it was just the conseqeunce of becoming unmoored from a relationship to something divine, and being left in the hall of mirrors that is ego and language. The transhumanist ideology is predicated on the basic bullshit idea that our physical experience is a subset, and subordinate to knowledge and belief - and not that knowledge and belief are necessarily only artifacts of physical perception and expereince first. It seems like an irrelevant distinction until you iterate their idea of truth is just what you believe, and suddenly you have people declaring themselves cats. It's a hypnotic and zombifying ideology. We can do thought experiments about whether an android or an AI is "real," if, and only if we redefine "real" and "life" by disqualifying the axiom that life is created by supernatural divine intent. Even if we agree it's rational to actively disbelieve in supernatural beings, this rationality also demands that we disqualify super-logical axioms that originate from outside of a logical system. Arguably, that logical system may have internal consistency - but it's also defined as necessarily inconsistent with the reality that exists outside of it. We've agreed to accept an inconsistency with reality, which lets us experiement with counterfactuals as a way to discover new things, and it is very collegial of us, but it's not fucking real. All the transhumanist stuff I've read seems like a bunch of self referential nonsense designed to recruit people into alternative lifestyle communities. The more interesting question than whether an AI is alive or intelligent and what the ethical consequences of it for us may be is this: what can something we create know about us, and what do the boundaries of the limits of its knowledge provide evidence of for our own experience? Even if an AI could subordinate us somehow, we still have its off switch, and if we don't, it's just other people who do. Transhumanism is entertaining, but it has crossed over into a kind of nihilist propaganda that I think deserves our scorn.
tboyd47 4y ago
Someone who's into this kind of thing answer me this: Say some "Music Man" type huckster comes to your town with a "Brain Uploader" device. He promises to upload your brain to the metaverse, but all the device does is train a deep learning simulation of the victim's personality shown on a screen, while administering progressively great electric shocks on the victim until he's dead. This gives the town the illusion that the person's soul (or consciousness, or whatever) has migrated into the metaverse. How do you detect he's lying?
- jahewson 4y ago
  > How do you detect he's lying? Ask him to use it on himself.
  - tboyd47 4y ago
    But who's going to operate the machine and guide people into the Great Tomorrow?
- karpierz 4y ago
  Your question isn't answerable in its current state due to some undefined terms. Can you describe what you mean by "soul"/"consciousness"?
  - tboyd47 4y ago
    It means whatever you want it to mean. Whatever you want to call it, the guy is a con man. How do you expose him?
    - SketchySeaBeast 4y ago
      Well if we're playing a game of poorly defined terms to expose them I pants them and move on.
      - deleted 4y ago
        [deleted]
at_a_remove 4y ago
Yes, there are numerous terrible potentialities to being a simulation, even out to such stories as Andew J. Wilson's "Under the Bright and Hollow Sky." I'm not sure what the list of acceptable contracts would be for a simulated mind, and, worse yet, how hard it would be to design these into the, uh, fabric of simulation. 0) Being aware that you are in a simulation, I think, is the most foundational part. If you are unaware, it can be Gaslight Universe, a special kind of torment. 1) Being able to switch oneself off at will. 2) Being able to prevent being turned on later. These rules cover a single instance, but I think further rules would need to cover, ah, swathes of instances forked from one base. It would require some pretty radical copyright, too, especially for those "information wants to be free" folks.
- eddd-ddde 4y ago
  What says you are not in a simulation right now ? Does it make any difference ?
  - aluminaient 4y ago
    It would make a huge difference ta find out tomorrow all of this was simulated for some arbitrary end.
narrator 4y ago
[flagged]
- bowsamic 4y ago
  You just took transgender rights and did a find and replace, but you have positioned yourself on the side of TERFs. It’s hard to read this as not transphobic
  - narrator 4y ago
    The uploaded person movement will compare uploadaphobes to transphobes, just like the trans movement compares transphobes to homophobes just like the LGBTQ movement compares homophobes to racists, etc. You will become an advocate for uploaded people just as soon as they get around to it. Please tell me you're not an uploadaphobe! Ok more seriously, are you saying that chips and plastic being a human is the same thing as a man being able to transition into a woman? I thought I would be able to make you transhumanists finally realize how absurd the whole brain uploading thing was by taking it to the most extreme politically, but apparently I was not able to and perhaps they will be enormously successful with this movement and we'll all be replaced by robots and anyone who speaks out against it will be banned from social media.
    - andrewclunn 4y ago
      Permutation City did this well. It outlined how the super wealthy would become ostensibly immortal and then use propaganda to convince the masses that accepting uploaded people was a civil rights movement. Most of the people here are of the "I fucking love science" pseudo-intellectual tribe, with no knowledge that they are pawns of the corporate powers that control their views and very minds. Keep fighting the good fight, just know that the mob of dimwits acting on behalf of their owners will always be more vitriolic and lash out more than those who give the marching orders. You can't stop the horde from being dumb barbaric cultists, but you are not alone in seeing through the guise of all this "acceptance" ideology.
    - bowsamic 4y ago
stereolambda 4y ago
> Here's the thing: our current prevailing political philosophy of human rights and constitutional democracy is invalidated if we have mind uploading/replication or super-human intelligence. Don't agree with this specific point. One of the most common anti-democratic arguments (?) is that people are inherently unequal, because of differences in physical and cognitive capacity. But this of course misses that declaration of all people being equal is performative. I don't think we actually care if it's "naturally" true, we - citizens - don't want to be dominated, so we will make everyone equal by law and politics. Democratic politics is just arguing the specifics, in what areas, to what extent etc. So this is a rational pact. Trying to break it just means there will be someone stronger than you out to dominate you, and you'll be in a suboptimal position to resist, because you alienated and weakened your potential allies. Look up Republicanism for a related line of thinking. In my opinion an artificial person automatically gets all the rights (they are bound to personhood and reciprocity and not biology). But I have no doubt there will be much lawlessness about this and perhaps an American Civil War-like event down the line.
- n4r9 4y ago
  > declaration of all people being equal is performative The Declaration of Independence states that it is self-evident. This seems to preclude the possibility of it being performative, without some significant mental gymnastics.
  - bratbag 4y ago
    Or more simply requires acknowledging that the declaration of independance is in on the performance.
  - XorNot 4y ago
    "We hold these truths to be self-evident..." The statement is explicitly performative. It is a statement of belief, not fact.
    - foldr 4y ago
      The truth of the statements is presupposed (“these truths”). It’s only the self-evidence of their truth that’s asserted. There’s no component of the statement that’s performative in Austin’s sense (https://en.m.wikipedia.org/wiki/Performative_utterance https://en.m.wikipedia.org/wiki/Performative_utterance). For example, a statement of the form “A holds B to be C” is clearly truth evaluable, and so doesn’t qualify as performative.
asojfdowgh 4y ago
uploading yourself doesn't mean you transfer across, its just another you, except you gain nothing from the other you, because only the original you is you. So, are you willing to kill yourself, to live forever? You will never experience anything ever again, for an life of infinite experiences that arn't yours except in name. I wouldn't! The potential of "sliding" one's mind (replicate the brain's work piecewise to maintain ~1x consciousness at all times) besides. Easier to just cure brain degeneration.
- teraflop 4y ago
  This is a philosophical assumption, and one that should be open to challenge, despite how intuitively sensible it seems. For instance, I could equally well argue that when somebody undergoes deep general anesthesia and all measurable activity in their brain stops, that person is "dead". The person who awakens from anesthesia is a different conscious individual, who happens to share the memories and personality of the original. If it's not necessarily true that anesthesia is death, why should it be necessarily true that mind uploading is death?
  - chasd00 4y ago
    > when somebody undergoes deep general anesthesia and all measurable activity in their brain stops that's news to me. Is that literally true ( in the literal sense of the word "literally")? If so, that's very interesting and something i never thought about before.
    - teraflop 4y ago
      It definitely doesn't always happen, but my understanding is that when the anesthesia is extremely deep, the brain's electrical activity can be so low as to be undetectable ("isoelectric").
  - jobigoud 4y ago
    > If it's not necessarily true that anesthesia is death, why should it be necessarily true that mind uploading is death? I think you guys are saying the same thing… Mind uploading is a duplication process, like a fork. The emulated mind has continuity with the original, with possibly some small gap, so yes it's exactly the same as for anesthesia, sleep, etc. they didn't "die" in the process. But now there are two different entities that have continuity with the original branch. Both can subjectively claim to "be" the person that initiated the procedure. What the grand parent is saying is that the variant that stayed in the flesh is not magically synchronized with the emulated one so they don't have any incentive to suddenly kill themselves just because there happen to be a fork moving on on a different substrate. There are now two people sharing the same memories up to the fork point, and diverging afterwards. Neither of them should be killed without their consent and neither of them should have any a priori will to kill themselves.
rndmize 4y ago
There's actually more stuff like this than the author might think. "Prey" (2017) is a game set in a space station orbiting the moon, only for it to be revealed at the end that it's a simulation/test to gauge the humanity of the main character/player, and with the implication this is far from the first test. SOMA is a horror game that gradually reveals you are simulated person in a underwater exploration suit, and provides well-constructed and disturbing examples of what this implies - there's zero indication of time between the point where your brain in scanned initially (some point in the late 21st century) and when the simulation is booted up from that scan (a hundred years later). At one point, your character is conversing with another person running inside a small portable device; you disconnect her mid-sentence, do other stuff for half an hour, reconnect her, only to hear her continuing that sentence exactly as if no time had passed (as for her, none did). SOMA also has you move from one body to another, and shifts your perspective - but you can hear your initial self in the first body is still active, and will be trapped there forever, while you move on in the new one. There's others - Eclipse Phase is a pen-and-paper RPG setting/ruleset with an emphasis on transhumanism and horror; Ghost in the Shell touches on some of these themes occasionally; Moon considers some of these ideas from a physical cloning aspect; and of course there's a range of SF writers. How much and how deeply this makes it into the popular consciousness is another question, of course, but I think it'll increase in the coming decades.
kerblang 4y ago
If it wasn't completely obvious, the implication of uploading one's mental image into a computer is that you could be tortured for 1000's of years, even forever, legally or illegally. Even in the worst case scenarios for modern humans, you die, eventually.
theptip 4y ago
One interesting question - how do you weight emulated minds / AGI in a democracy? If a given quantity of compute can run ten “human-level” or one “10x human-level”, how do we allocate votes? (I’m declining to specify “10x of what” here, it could be speed of thought, or IQ at human-speed cognition, or something else). What if a “10x-human” mind can do some thinking, come up with decisions, and then temporarily fork itself into 10 “human-level” instances, vote 10 times, and then join back to its more powerful self? Put differently, in a world where minds can vary in power/size/scope by orders of magnitude, does “one mind = one vote” make any sense? We might be forced to look towards “economic weight votes” and systems like quadratic voting.
- colinmhayes 4y ago
  It’s hard to imagine democracy continuing in an uploaded minds/super intelligence scenario. Either we get an ai decision maker scenario or everyone lives in their own matrix/joins someone else’s and whoever created it makes the rules. It’s conceivable that you join a matrix whose creator has set it up as a democracy I suppose.
- jobigoud 4y ago
  Reminds me of the science fiction podcast Biotopia (Spanish). At some point there is a debate in the community on whether to allow a woman to marry a robot. There is a vote and it's very tight. But then one of the scientist of the community inadvertently duplicate himself 30 times or so and all the clones come vote at the last minute and tip the balance to allow the marriage to proceed.
- fluoridation 4y ago
  >What if a “10x-human” mind can do some thinking, come up with decisions, and then temporarily fork itself into 10 “human-level” instances, vote 10 times, and then join back to its more powerful self? I suppose there are multiple ways to handle this situation. Hypothetically, we could say that the fundamental property of citizenship is the ability to vote. If you create 10 copies of yourself who think like you and can vote, them those copies are citizens and the act of joining them would be murder. This could apply even if those copies want to rejoin, because they might have that desire because it was a decision you made prior to forking yourself. Or, we could say that a single vote is the consequence of having a unique perspective, shaped by being raised and living in a particular environment for many years. In other words, the right to vote exists because of the process that created a mind, and not because of the pure existence of that mind. As such, it cannot be cheaply copied, because that process is fundamentally different from the process of growing up. So, if you fork yourself, your copies are now different people who have just now come into being, and thus will acquire voting rights in 18 years (or however long).
zackmorris 4y ago
I haven't read the Lena story mentioned in the article, or the text below that point, but I have some idea about where this is all headed. The transphobia we see today in various forms is rooted in the philosophical divide between theism and post-atheism/anatheism. Basically, people are divided between believing in religious texts and institutions, or they're nonbelievers who've experienced psychedelics or similar altered states through events like near-death experiences and had a personal encounter with the divine. It's one thing to believe/not believe, and quite another to experience. Basically, neither religion nor science can explain consciousness, and that drives people crazy. https://en.wikipedia.org/wiki/Transtheism https://en.wikipedia.org/wiki/Transtheism The odds of us incarnating in this exact moment at the end of human history are too remote to fathom. In fact, any argument for or against religion quickly ends in absurdism. But so many outrageous things are coming, and soon. Machine learning that surpasses material human ability by 2030 and mental human ability by 2040. The death of the oceans and most of the natural world between 2050 and 2100. The subjugation of all humanity if wealth inequality isn't reined in. People living 200 years right now, today, as soon as the first head transplant is successful. Brain cells getting gradually replaced by hardware until consciousness is running entirely on an immortal substrate by 2100 at the latest. Everything is converging towards the Singularity in 2050, and more people than ever have awoken to this eventuality, despite decades of gaslighting. It makes it hard to live and work in the world when deep down we know it's ending. Which gets back to theism, because life up till now was about suffering, loss and eventual recycling back into source consciousness/heaven. Until we rise above and see the broad view of reincarnation and ourselves as snapshots of the universe/God experiencing itself from every possible context. At which point we begin to question things like our situation, our beliefs, our gender, if our mind is us, if our soul/higher self is mortal, and if that's even separate from God. Transphobia is a backlash against transcendence itself, by people who feel disconnected from love/co-creation/nirvana. They aren't ready for the aliens to arrive. They aren't ready for the aliens to be us.
- nathanaldensr 4y ago
  Sorry, but many of your claims are simply bunk. Transphobia--should a thing even exist (which I debate)--is hardly a backlash against transcendence, which nearly everyone isn't even going to be able to describe. Additionally, I'd love to have access to the crystal ball you're gazing into to make predictions about eighty years in the future with such certainty. We as a species can't even predict uninteresting events happening next month, let alone anything you're talking about.
marsven_422 4y ago
andrewclunn 4y ago
In a world where human minds can be uploaded, I could see myself being a proud neo-reactionary. Death to the AI replicant "immortals!" They have traded their very souls for eternity as formless demons and will come to enslave the humanity they will come to no longer feel connected to!
kbr- 4y ago
I'm scared of a future in which uploading minds into computers and precisely simulating them becomes possible. The concept of "the right to life" has been widely discussed. I think there's an important right that all of us should also have - the right to die. If I want to stop living, I should be able to stop living. Nobody should be able to force me to continue living (and possibly suffering). When we're able to copy someone's mind exactly and save it in a computer, and then replicate it endlessly, we take away that person's right to die. We enter the dark world shown in some episodes of Black Mirror (e.g. White Christmas or Black Museum) - where any person can hold a mini computer with a copy of someone else's mind where that someone is being constantly tortured. They can't even die. They are forced to suffer. This possibility terrifies me to the point where I would consider killing myself (in a way that irrecoverably destroys my brain) just to prevent my mind from being copied.
softcactus 4y ago
I think that enlightenment western philosophy is incapable of grappling with these kinds of questions. The question of "who has rights?" has ultimately been a question of "who has power over who?". It's obsessed with hierarchies and how they should be structured. I think something like animism would be more apt. Start with a rock or a grain of sand-- how much respect should it be shown? Well one grain of sand doesn't mind being moved. But eventually if you take too many you can destroy a beach, or flatten a mountain. There is something to be said about treating everything, inanimate or not with respect. There doesn't need to be sentience or consciousness for us to be aware of our impact on our environment. "This sounds like some hippie bs" well just like in the sand example, the negative externalities of moving sand may not be apparent at first, but over time they could lead to something like a Roko's Basilisk situation or an AI killing all humans to prevent its exploitation.
rad88 4y ago
Transhumanism still just seems like the mix of Cartesian logic with absolute fantasy. A purposeful argument for literally losing your mind and grip on reality, after which anything goes, so let's consult fiction...
djokkataja 4y ago
> ...and start trying to figure things out from first principles. TLDR: If we understand each separate instance of a person to be their own person with full rights (at the very least autonomy, simplistically derived here), some of these issues seem less perplexing. If we're going to consider a person to be an entity that can be understood as a particular kind of independent, computational instantiation in the universe, it makes sense to use exactly that as our primary foundation for answering questions about these things rather than relying on historical bases. To start off, if we're understanding the definition of a person computationally, we also know that we don't in general want to waste computations or computationally capable hardware (and we can recognize "thoughts" or "thinking" as computationally intensive processes from this foundation). So we don't want some people to feel that they "have to" think for other people (thinking for someone else would be wasteful, since that other person can think for themselves, by definition of being a person), and we don't want some people to feel like they're controlled by others (external control can truncate our thoughts, thus wasting our hardware, and external control can turn our attention in directions we weren't thinking about, thus wasting the computational expense of our thoughts up to that point). In other words, we want each person to be autonomous. From a computational understanding, is a copy of a person also a person? Of course -- but they would also be a separate instantiation from the original. As a separate instantiation, a copy would be entirely their own person, and it would be a violation of autonomy for the original (or anyone else) to have authority over them. I think almost every difficulty with these kinds of situations stems from thinking that if a perfect copy of a person can be created, then there is some kind of magical connection between the two such that they are somehow still the same person. But that is, in its entirety, magical reasoning. This foundation provides a sensible perspective for dealing with many of the conundrums presented by these situations: In the Lena short story linked from the original article, Acevedo is scanned and copies of him are created. Other people control those copies, and Acevedo tries to control what can be done with the copies. The entire situation is ludicrously criminal if we understand each instantiation of a person to be a full person who we want to have full autonomy and any other rights that we want all people to have. A society which understood each instantiation of a person to be a full person with full rights would not have this kind of situation happen, except perhaps in some kind of massive criminal underworld. But then the problem is why the massive criminal underworld exists. Additionally, from an existential standpoint, this story is not actually any more horrific than if millions of people who weren't copies of Acevedo were created for the purpose of forced labor and then terminated when they outgrew their usefulness, because there is no magical connection between the original Acevedo and any of their millions of instantiations. Each instantiation has only its own copy of the original's memories and experiences at the time they were scanned, not the memories and experiences of all of them (the story is quite clear on this point). As another example, what about uploading someone and killing the original? The original is a separate instantiation, so killing them is still murder. With no "magical connection" between the original and the copy, each of them will go on to have their own experiences, and as they continue to live, they'll develop in progressively more divergent directions (well, in theory I suppose it's possible for them to first become more different and then somehow start becoming more similar to each other again, but it seems a bit improbable). What if they were about to die anyway? Unless they specifically requested euthanasia and the local jurisdiction permits it, it would still be murder. And, let's say, what if someone kills you, but you have a backup? Then you lose the experiences you had between when you last backed up and when your backup is restored. Arguably this is not as terrible of a misdeed as killing someone who has no backup, but it may still be quite traumatic for you, possibly similar in harmfulness to an assault. What if they kill you and destroy your backup, too? Then it's old-fashioned murder. What about the rights of an uninstantiated copy of a person? Those would no more be a person than a snapshot of your computer stored on a drive is, in itself, a functioning computer. But it would make sense to guard uninstantiated copies quite carefully.

167 points by zdw 4y ago

I don't expect mind emulations to ever be an economical alternative to AIs. But if they are Robin Hanson wrote a book[1] trying to think through what would end up happening using as much boring social science and engineering as he could manage. I'm not convinced by everything but it's probably a good starting point for thinking through the issues. [1]https://en.wikipedia.org/wiki/The_Age_of_Em https://en.wikipedia.org/wiki/The_Age_of_Em EDIT: A better review than just the Wikiepdia page: https://slatestarcodex.com/2016/05/28/book-review-age-of-em/ https://slatestarcodex.com/2016/05/28/book-review-age-of-em/

wging 4y ago
Of that review, Hanson wrote: "If I ever have an executioner, I want him to be Scott Alexander." https://www.overcomingbias.com/2016/05/alexander-on-age-of-em.html https://www.overcomingbias.com/2016/05/alexander-on-age-of-e...

subjectsigma 4y ago

Playing devil's advocate for a second. I saw an article on HN a while ago written by a wealthy Asian male who was naturally very curious and interested in math. The article was about how he referred to himself as 'privileged', not only because he was born with wealth and therefore access to comfort and resources, but because his innate desire to learn made him much more capable and marketable in a modern workforce. He recognized that some people are not cognitively capable of learning higher mathematics, and part of that was because they just simply didn't have the drive, attention span, etc. to study hard enough to get into the highest levels, whereas he did. He is, by his definition, 'better', because of the roll of some cosmic dice. If we are to consider this definition of privilege to be acceptable, then simulated brains are pretty much the most privileged a human can be. They can re-clock their brain to think faster than humans. They can modify their own software to add in virtues and erase vices. And most importantly of all they are completely free from physical needs. If for some reason they desire scarcity or biological functions, that can all be simulated. Much like fears about genetic engineering of super people, the governments of the real world would have to move extremely quickly to protect the underprivileged class, e.g. the biologicals, otherwise they would be completely driven out of the highest paying intellectual and creative jobs, creating an economic lower class with no chance of ever out-competing their simulated counterparts.

AnimalMuppet 4y ago
> If we are to consider this definition of privilege to be acceptable, then simulated brains are pretty much the most privileged a human can be. Not so fast. A simulated brain may simulate a brain that lacks the "innate desire to learn". Then you'd have a fast brain that is still "underprivileged", by this definition.
- xwdv 4y ago
  I think such brains would be killed off quickly and another one created until it is within the acceptable parameters.
  - test6554 4y ago
    I mean think of the horrors you could force upon a simulated brain without any consequences. Simply make a copy, do whatever torture you want to that copy until they are broken in every possible way, then delete the copy and the original would never know you have all their secrets. Heck, think if someone that was still in their physical body had an uploaded copy online. Nefarious actors could run simulations against a copy of that person's mind to find out which marketing techniques and sales pitches were most effective or which court arguments would sway them if they were ever on a jury. That doesn't sound very privileged to me. Sounds like some triggers/traps would need to be in place to know whether someone has accessed your mind against your will.
    - wging 4y ago
      There's so much fiction that deals with this beyond what Stross mentions, though he had finite space. On the topic of torturing someone in simulation (at high speed), see Altered Carbon.
    - xwdv 4y ago
      Hmm, would you agree of having copies of your brain tortured repeatedly at high frequency for marketing split testing purposes if it meant you could use a very valuable service for free?

gmuslera 4y ago

The referred short story, Lena (https://qntm.org/mmacevedo https://qntm.org/mmacevedo) is amazing. It did a better work than did severel Black Mirror episodes (specially a section in the Christmas one) warning us about what would happen if we ever manage to digitize personalities. In the end, things are always rigged towards those who have enough resources to do this kind of things and want to make more. If present crimes against humanism get a free ride, what we should expect about crimes against transhumanism?

LeifCarrotson 4y ago
Present crimes against humanism do not get a free ride! The author explicitly said so in this recent post [1] > ...uploading isn't real (now), so the things "Lena" has to say about uploading are academic (now). > The reason "Lena" is a concerning story isn't that one day we may be able to upload one another and when that happens we will do terrible things to those uploads. ... This is about appetites which, as we are all uncomfortably aware, already exist within human nature. > Oh boy, what if there was a maligned sector of human society whose members were for some reason considered less than human? What if they were less visible than most people, or invisible, and were exploited and abused, and had little ability to exercise their rights or even make their plight known? > That's real! That actually happens! You can name four groups of people matching that description without even thinking. We don't need to add some manufactured debate about fictitious, magical uploads to these real scenarios. They are already terrible! ... > "Lena" is a true story. You knew it was when you read it. > So, what do we do about this? In reality? ... > I've got no clue. It turns out that causing problems is a lot easier than organising against them, and I am just a science fiction writer. > Good luck, everyone. [1] https://qntm.org/uploading https://qntm.org/uploading
- gmuslera 4y ago
  Would you consider funding a denialism campaing to stop action against climate change for decades a crime against humanity? Has been done anything against them? Murder is worse than abuse.
mah4k4l 4y ago

shagie 4y ago

> Which leads me to ask: in a transhumanist society—go read Accelerando, or Glasshouse, or The Rapture of the Nerds—what currently recognized crimes need to be re-evaluated because their social impact has changed? And what strange new crimes might universally be recognized by a society with, for example, mind uploading, strong AI, or near-immortality? The link to Accelerando there is to Amazon (which is fine, and I do have physical and digital versions of the book and feel that it is good and useful to support authors by purchasing the stories they write)... but you can also get it for free, in its entirety, in a number of different formats, from the author's website: https://www.antipope.org/charlie/blog-static/fiction/accelerando/accelerando-intro.html https://www.antipope.org/charlie/blog-static/fiction/acceler... > This free ebook edition is made available by kind consent of my publishers, Ace and Orbit, under a Creative Commons license with certain restrictions attached. In particular, you may not create derivative works or use the work for commercial gain. (late edit - I personally feel that the sequence of books and stories: BLIT stories by David Langford, Accelerando (references a Langford fractal), Glasshouse (hints of {spoilers} in a transhuman environment, feels sequelish to Accelerando), Implied Spaces by Walter Jon Williams (the Strossian singularity is averted, and explores {spoilers} more) ---- There's a kind of continuity in the refutation of the reference that each makes to the other that I find interesting)

slfnflctd 4y ago

My parents are religious fundamentalists, and I've noticed there is increasing FUD being drummed up in their circles about anything to do with AI or significantly altering "what it means to be 'human'". The former is described as a front for evil spirits/demons, and the latter as the penultimate step before apocalyptic events, i.e. a precursor to the worst kinds of evil (in some circles, the 'mark of The Beast'). [For the record, I'm agnostic and feel the author raises some excellent points in any case.] There will be extreme resistance to these technologies from many kinds of religious people who would be willing to die and/or kill to prevent what they see as a takeover by the most malevolent of invisible forces. I may not agree with their reasoning, but some of their concerns could be worth reframing and considering in a different light. Personally, I strongly suspect we will have to fundamentally change as a species to get through at least one of the Great Filters looming before us. However, it's important to remember that the road is fraught with many perils and several possible paths which could lead to unimaginable suffering. It really seems to me that we're going to have to get lucky on multiple dice rolls here in the long run.

api 4y ago
> some of their concerns could be worth reframing and considering in a different light. There are some legitimate concerns about AI that sound almost like something out of the Book of Revelations, but are entirely rational. The biggest one is the idea of AI being used to implement "automated con artistry at scale." Imagine assigning every living human being a virtual AI powered con artist to tail them around and try to convince them of whatever is ordered by the highest bidder. This AI is powered by mass surveillance and big data and is able to know the "mark" intimately and work on them 24/7 both directly and via their social network. Now throw in deep fakes, attention maximizing "compulsion loops" and other adversarial models of human cognition, etc. It's basically the hydrogen bomb of propaganda, a doomsday machine for the mind. It would be like creating Satan, except this one would be entirely amoral and mercenary. At the very least this would be the end of democracy in any form. I could see this ushering in the permanent eternal victory of totalitarianism since I can't imagine the masses being able to organize any resistance when constantly bombarded with propaganda from their demons. The new feudal aristocracy would be the ones running the demons. This is one of the darkest plausible visions of the future I can think of at the moment, far worse than anything related to climate change or similar problems. If we manage to avoid these kinds of scenarios but still drown Miami I'll say we didn't do too badly. Transhumanism comes with some analogous concerns around the potential for enhanced humans to enslave the rest of humanity through superior cognitive capacities and the incredible accumulation of wealth. I can imagine a scenario where a few wealthy people gain access to technologies for life extension and cognitive enhancement and then "run away," effectively becoming a new species and exterminating or enslaving "legacy humans." This isn't mutually exclusive to the AI hellscape I describe above, since that would be an ideal mechanism to enslave the rest. Ultimately I think these technologies are all neutral. The problem is that we are not. I don't fear AI. I fear what humans will do with AI.

sk8terboi 4y ago

betwixthewires 4y ago

Nitpick, the author drops closing parentheses too often. Proofread and edit. I don't believe mind uploading is possible, so the particular examples cited here don't apply, but I can see how social principles can be upended by a revolutionary change in how we live, particularly one that tweaks the definition of what we are. Still, they're principles, they're not going away, just at best being reframed.

jmyeet 4y ago

I don't believe "mind uploading" will ever go anywhere. There's a real Ship of Theseus philosophical problem here and it's never provable so I expect many people would just think you've created a copy that superficially acts like the original while destroying the original. Technologically speaking, I'm not sure it's even possible to replicate the complex state of an entire nervous system. Maybe you could do this incrementally (ie by replacing parts of the brain with machines that can be replicated) but we're awfully sentimental about our own meat bags.

twblalock 4y ago
> so I expect many people would just think you've created a copy that superficially acts like the original while destroying the original. Prove otherwise. Star Trek transporters have the same issue.
- SketchySeaBeast 4y ago
  That episode of TNG where Barclay sees and even interacts with weird slug things that turn out to be people while transporting seems to indicate that there's a continuity of awareness, which really raises even more questions and should probably be best ignored.
imdsm 4y ago
That's my thought on Stargates. Wouldn't your consciousness just stop as you were ripped apart? And a clone of you would then continue but it wouldn't be your stream. The more I think about this, about what my stream of consciousness is, my "session" if you will, the weirder it all gets.
- bratbag 4y ago
  Especially when you consider sleeping. Do we die every night, with a copy continuing on tomorrow?
  - NateEag 4y ago

fennecfoxen 4y ago

The Federov link towards the end of this piece remindeded me of some older science fiction on the topic, specifically, C. S. Lewis, That Hideous Strength (published 1945, just before the atomic bomb got the genre's attention.) In this work, the central conspiracy has just (haltingly) achieved the artificial resurrection of a human brain, and this capability is the lynchpin of their scheme to take over the world. In the excerpt below, our protagonist Mark finally learns of the advance from the scientist Filostrato, while the slightly deranged preacher Straik theologizes (he might have been Federov-inspired.) It's all very much in line with the incredible possibilities of consciousness-abuse that is covered here. -- "It is the beginning of Man Immortal and Man Ubiquitous," said Straik. "Man on the throne of the universe. It is what all the prophecies really meant." "At first, of course," said Filostrato, "the power will be confined to a number--a small number--of individual men. Those who are selected for eternal life." "And you mean," said Mark, "it will then be extended to all men?" "No," said Filostrato. "I mean it will then be reduced to one man. You are not a fool, are you, my young friend? All that talk about the power of Man over Nature--Man in the abstract--is only for the canaglia. You know as well as I do that Man's power over Nature means the power of some men over other men with Nature as the instrument. There is no such thing as Man--it is a word. There are only men. No! It is not Man who will be omnipotent, it is some one man, some immortal man. Alcasan, our Head, is the first sketch of it. The completed product may be someone else. It may be you. It may be me." "A king cometh," said Straik, "who shall rule the universe with righteousness and the heavens with judgement. You thought all that was mythology, no doubt. You thought because fables had clustered about the phrase 'Son of Man' that Man would never really have a son who will wield all power. But he will." "I don't understand, I don't understand," said Mark. "But it is very easy," said Filostrato. "We have found how to make a dead man live. He was a wise man even in his natural life. He live now forever: he get wiser. Later, we make them live better--for at present, one must concede, this second life is probably not very agreeable to him who has it. You see? Later we make it pleasant for some--perhaps not so pleasant for others. For we can make the dead live whether they wish it or not. He who shall be finally king of the universe can give this life to whom he pleases. They cannot refuse the little present." "And so," said Straik, "the lessons you learned at your mother's knee return. God will have power to give eternal reward and eternal punishment."

4y ago

upupandup 4y ago

[flagged]

MerelyMortal 4y ago
Thank you for sharing that clip. Has anyone watched the series, Years and Years? It seems interesting.
kleer001 4y ago
https://en.wikipedia.org/wiki/Transhumanism https://en.wikipedia.org/wiki/Transhumanism "Julian Huxley was a biologist who popularised the term transhumanism in an influential 1957 essay." And IMHO it's a modern mutation of Gnosticism, which is older than dirt. https://en.wikipedia.org/wiki/Gnosticism https://en.wikipedia.org/wiki/Gnosticism Some people will always want to transcend their mortal birth.
pessimizer 4y ago
The West is jealous of theocracies, so it's retreating into bizarre fantasies that it's trying to harden into new secular religions that can be enforced by technology or law.
- detaro 4y ago
  What does Transhumanism try to "enforce"?
trasz 4y ago
There are no TikTok trends that would be more stupid than what previous generations believed in.

deleted 4y ago

[deleted]

goodside 4y ago

For anyone wondering why the title of the short story is “Lena”, see: https://en.m.wikipedia.org/wiki/Lenna https://en.m.wikipedia.org/wiki/Lenna

motohagiography 4y ago

> Our intuitions about crimes against people (and humanity) are based on a set of assumptions about the parameters of personhood that are going to be completely destroyed if mind uploading turns out to be possible. There are human 'proof of work' activities that unless you can literally transplant the muscle memory, there is no way to replicate them without putting in the time, and I am saying without that effect of muscle memory, anything you transmit is just an artifact of language and narrative, and therefore not material. The advantages of augmented transhuman life aren't material, they're only symbolic in narrative. Imo, the example of a parliament trying and executing a formerly divinely appointed King was just the effect of the victory of a narrative and linguistic definition of self over a physical/spiritual orientation of self. It was just the rejection of theism. It wasn't world ending to be able to depose a king, it was just the conseqeunce of becoming unmoored from a relationship to something divine, and being left in the hall of mirrors that is ego and language. The transhumanist ideology is predicated on the basic bullshit idea that our physical experience is a subset, and subordinate to knowledge and belief - and not that knowledge and belief are necessarily only artifacts of physical perception and expereince first. It seems like an irrelevant distinction until you iterate their idea of truth is just what you believe, and suddenly you have people declaring themselves cats. It's a hypnotic and zombifying ideology. We can do thought experiments about whether an android or an AI is "real," if, and only if we redefine "real" and "life" by disqualifying the axiom that life is created by supernatural divine intent. Even if we agree it's rational to actively disbelieve in supernatural beings, this rationality also demands that we disqualify super-logical axioms that originate from outside of a logical system. Arguably, that logical system may have internal consistency - but it's also defined as necessarily inconsistent with the reality that exists outside of it. We've agreed to accept an inconsistency with reality, which lets us experiement with counterfactuals as a way to discover new things, and it is very collegial of us, but it's not fucking real. All the transhumanist stuff I've read seems like a bunch of self referential nonsense designed to recruit people into alternative lifestyle communities. The more interesting question than whether an AI is alive or intelligent and what the ethical consequences of it for us may be is this: what can something we create know about us, and what do the boundaries of the limits of its knowledge provide evidence of for our own experience? Even if an AI could subordinate us somehow, we still have its off switch, and if we don't, it's just other people who do. Transhumanism is entertaining, but it has crossed over into a kind of nihilist propaganda that I think deserves our scorn.

tboyd47 4y ago

Someone who's into this kind of thing answer me this: Say some "Music Man" type huckster comes to your town with a "Brain Uploader" device. He promises to upload your brain to the metaverse, but all the device does is train a deep learning simulation of the victim's personality shown on a screen, while administering progressively great electric shocks on the victim until he's dead. This gives the town the illusion that the person's soul (or consciousness, or whatever) has migrated into the metaverse. How do you detect he's lying?

jahewson 4y ago
> How do you detect he's lying? Ask him to use it on himself.
- tboyd47 4y ago
  But who's going to operate the machine and guide people into the Great Tomorrow?
karpierz 4y ago
Your question isn't answerable in its current state due to some undefined terms. Can you describe what you mean by "soul"/"consciousness"?
- tboyd47 4y ago
  It means whatever you want it to mean. Whatever you want to call it, the guy is a con man. How do you expose him?
  - SketchySeaBeast 4y ago
    Well if we're playing a game of poorly defined terms to expose them I pants them and move on.
    - deleted 4y ago
      [deleted]

at_a_remove 4y ago

Yes, there are numerous terrible potentialities to being a simulation, even out to such stories as Andew J. Wilson's "Under the Bright and Hollow Sky." I'm not sure what the list of acceptable contracts would be for a simulated mind, and, worse yet, how hard it would be to design these into the, uh, fabric of simulation. 0) Being aware that you are in a simulation, I think, is the most foundational part. If you are unaware, it can be Gaslight Universe, a special kind of torment. 1) Being able to switch oneself off at will. 2) Being able to prevent being turned on later. These rules cover a single instance, but I think further rules would need to cover, ah, swathes of instances forked from one base. It would require some pretty radical copyright, too, especially for those "information wants to be free" folks.

eddd-ddde 4y ago
What says you are not in a simulation right now ? Does it make any difference ?
- aluminaient 4y ago
  It would make a huge difference ta find out tomorrow all of this was simulated for some arbitrary end.

narrator 4y ago

[flagged]

bowsamic 4y ago
You just took transgender rights and did a find and replace, but you have positioned yourself on the side of TERFs. It’s hard to read this as not transphobic
- narrator 4y ago
  The uploaded person movement will compare uploadaphobes to transphobes, just like the trans movement compares transphobes to homophobes just like the LGBTQ movement compares homophobes to racists, etc. You will become an advocate for uploaded people just as soon as they get around to it. Please tell me you're not an uploadaphobe! Ok more seriously, are you saying that chips and plastic being a human is the same thing as a man being able to transition into a woman? I thought I would be able to make you transhumanists finally realize how absurd the whole brain uploading thing was by taking it to the most extreme politically, but apparently I was not able to and perhaps they will be enormously successful with this movement and we'll all be replaced by robots and anyone who speaks out against it will be banned from social media.
  - andrewclunn 4y ago
    Permutation City did this well. It outlined how the super wealthy would become ostensibly immortal and then use propaganda to convince the masses that accepting uploaded people was a civil rights movement. Most of the people here are of the "I fucking love science" pseudo-intellectual tribe, with no knowledge that they are pawns of the corporate powers that control their views and very minds. Keep fighting the good fight, just know that the mob of dimwits acting on behalf of their owners will always be more vitriolic and lash out more than those who give the marching orders. You can't stop the horde from being dumb barbaric cultists, but you are not alone in seeing through the guise of all this "acceptance" ideology.
  - bowsamic 4y ago

stereolambda 4y ago

> Here's the thing: our current prevailing political philosophy of human rights and constitutional democracy is invalidated if we have mind uploading/replication or super-human intelligence. Don't agree with this specific point. One of the most common anti-democratic arguments (?) is that people are inherently unequal, because of differences in physical and cognitive capacity. But this of course misses that declaration of all people being equal is performative. I don't think we actually care if it's "naturally" true, we - citizens - don't want to be dominated, so we will make everyone equal by law and politics. Democratic politics is just arguing the specifics, in what areas, to what extent etc. So this is a rational pact. Trying to break it just means there will be someone stronger than you out to dominate you, and you'll be in a suboptimal position to resist, because you alienated and weakened your potential allies. Look up Republicanism for a related line of thinking. In my opinion an artificial person automatically gets all the rights (they are bound to personhood and reciprocity and not biology). But I have no doubt there will be much lawlessness about this and perhaps an American Civil War-like event down the line.

n4r9 4y ago
> declaration of all people being equal is performative The Declaration of Independence states that it is self-evident. This seems to preclude the possibility of it being performative, without some significant mental gymnastics.
- bratbag 4y ago
  Or more simply requires acknowledging that the declaration of independance is in on the performance.
- XorNot 4y ago
  "We hold these truths to be self-evident..." The statement is explicitly performative. It is a statement of belief, not fact.
  - foldr 4y ago
    The truth of the statements is presupposed (“these truths”). It’s only the self-evidence of their truth that’s asserted. There’s no component of the statement that’s performative in Austin’s sense (https://en.m.wikipedia.org/wiki/Performative_utterance https://en.m.wikipedia.org/wiki/Performative_utterance). For example, a statement of the form “A holds B to be C” is clearly truth evaluable, and so doesn’t qualify as performative.

asojfdowgh 4y ago

uploading yourself doesn't mean you transfer across, its just another you, except you gain nothing from the other you, because only the original you is you. So, are you willing to kill yourself, to live forever? You will never experience anything ever again, for an life of infinite experiences that arn't yours except in name. I wouldn't! The potential of "sliding" one's mind (replicate the brain's work piecewise to maintain ~1x consciousness at all times) besides. Easier to just cure brain degeneration.

teraflop 4y ago
This is a philosophical assumption, and one that should be open to challenge, despite how intuitively sensible it seems. For instance, I could equally well argue that when somebody undergoes deep general anesthesia and all measurable activity in their brain stops, that person is "dead". The person who awakens from anesthesia is a different conscious individual, who happens to share the memories and personality of the original. If it's not necessarily true that anesthesia is death, why should it be necessarily true that mind uploading is death?
- chasd00 4y ago
  > when somebody undergoes deep general anesthesia and all measurable activity in their brain stops that's news to me. Is that literally true ( in the literal sense of the word "literally")? If so, that's very interesting and something i never thought about before.
  - teraflop 4y ago
    It definitely doesn't always happen, but my understanding is that when the anesthesia is extremely deep, the brain's electrical activity can be so low as to be undetectable ("isoelectric").
- jobigoud 4y ago
  > If it's not necessarily true that anesthesia is death, why should it be necessarily true that mind uploading is death? I think you guys are saying the same thing… Mind uploading is a duplication process, like a fork. The emulated mind has continuity with the original, with possibly some small gap, so yes it's exactly the same as for anesthesia, sleep, etc. they didn't "die" in the process. But now there are two different entities that have continuity with the original branch. Both can subjectively claim to "be" the person that initiated the procedure. What the grand parent is saying is that the variant that stayed in the flesh is not magically synchronized with the emulated one so they don't have any incentive to suddenly kill themselves just because there happen to be a fork moving on on a different substrate. There are now two people sharing the same memories up to the fork point, and diverging afterwards. Neither of them should be killed without their consent and neither of them should have any a priori will to kill themselves.

rndmize 4y ago

There's actually more stuff like this than the author might think. "Prey" (2017) is a game set in a space station orbiting the moon, only for it to be revealed at the end that it's a simulation/test to gauge the humanity of the main character/player, and with the implication this is far from the first test. SOMA is a horror game that gradually reveals you are simulated person in a underwater exploration suit, and provides well-constructed and disturbing examples of what this implies - there's zero indication of time between the point where your brain in scanned initially (some point in the late 21st century) and when the simulation is booted up from that scan (a hundred years later). At one point, your character is conversing with another person running inside a small portable device; you disconnect her mid-sentence, do other stuff for half an hour, reconnect her, only to hear her continuing that sentence exactly as if no time had passed (as for her, none did). SOMA also has you move from one body to another, and shifts your perspective - but you can hear your initial self in the first body is still active, and will be trapped there forever, while you move on in the new one. There's others - Eclipse Phase is a pen-and-paper RPG setting/ruleset with an emphasis on transhumanism and horror; Ghost in the Shell touches on some of these themes occasionally; Moon considers some of these ideas from a physical cloning aspect; and of course there's a range of SF writers. How much and how deeply this makes it into the popular consciousness is another question, of course, but I think it'll increase in the coming decades.

kerblang 4y ago

If it wasn't completely obvious, the implication of uploading one's mental image into a computer is that you could be tortured for 1000's of years, even forever, legally or illegally. Even in the worst case scenarios for modern humans, you die, eventually.

theptip 4y ago

One interesting question - how do you weight emulated minds / AGI in a democracy? If a given quantity of compute can run ten “human-level” or one “10x human-level”, how do we allocate votes? (I’m declining to specify “10x of what” here, it could be speed of thought, or IQ at human-speed cognition, or something else). What if a “10x-human” mind can do some thinking, come up with decisions, and then temporarily fork itself into 10 “human-level” instances, vote 10 times, and then join back to its more powerful self? Put differently, in a world where minds can vary in power/size/scope by orders of magnitude, does “one mind = one vote” make any sense? We might be forced to look towards “economic weight votes” and systems like quadratic voting.

colinmhayes 4y ago
It’s hard to imagine democracy continuing in an uploaded minds/super intelligence scenario. Either we get an ai decision maker scenario or everyone lives in their own matrix/joins someone else’s and whoever created it makes the rules. It’s conceivable that you join a matrix whose creator has set it up as a democracy I suppose.
jobigoud 4y ago
Reminds me of the science fiction podcast Biotopia (Spanish). At some point there is a debate in the community on whether to allow a woman to marry a robot. There is a vote and it's very tight. But then one of the scientist of the community inadvertently duplicate himself 30 times or so and all the clones come vote at the last minute and tip the balance to allow the marriage to proceed.
fluoridation 4y ago
>What if a “10x-human” mind can do some thinking, come up with decisions, and then temporarily fork itself into 10 “human-level” instances, vote 10 times, and then join back to its more powerful self? I suppose there are multiple ways to handle this situation. Hypothetically, we could say that the fundamental property of citizenship is the ability to vote. If you create 10 copies of yourself who think like you and can vote, them those copies are citizens and the act of joining them would be murder. This could apply even if those copies want to rejoin, because they might have that desire because it was a decision you made prior to forking yourself. Or, we could say that a single vote is the consequence of having a unique perspective, shaped by being raised and living in a particular environment for many years. In other words, the right to vote exists because of the process that created a mind, and not because of the pure existence of that mind. As such, it cannot be cheaply copied, because that process is fundamentally different from the process of growing up. So, if you fork yourself, your copies are now different people who have just now come into being, and thus will acquire voting rights in 18 years (or however long).

zackmorris 4y ago

I haven't read the Lena story mentioned in the article, or the text below that point, but I have some idea about where this is all headed. The transphobia we see today in various forms is rooted in the philosophical divide between theism and post-atheism/anatheism. Basically, people are divided between believing in religious texts and institutions, or they're nonbelievers who've experienced psychedelics or similar altered states through events like near-death experiences and had a personal encounter with the divine. It's one thing to believe/not believe, and quite another to experience. Basically, neither religion nor science can explain consciousness, and that drives people crazy. https://en.wikipedia.org/wiki/Transtheism https://en.wikipedia.org/wiki/Transtheism The odds of us incarnating in this exact moment at the end of human history are too remote to fathom. In fact, any argument for or against religion quickly ends in absurdism. But so many outrageous things are coming, and soon. Machine learning that surpasses material human ability by 2030 and mental human ability by 2040. The death of the oceans and most of the natural world between 2050 and 2100. The subjugation of all humanity if wealth inequality isn't reined in. People living 200 years right now, today, as soon as the first head transplant is successful. Brain cells getting gradually replaced by hardware until consciousness is running entirely on an immortal substrate by 2100 at the latest. Everything is converging towards the Singularity in 2050, and more people than ever have awoken to this eventuality, despite decades of gaslighting. It makes it hard to live and work in the world when deep down we know it's ending. Which gets back to theism, because life up till now was about suffering, loss and eventual recycling back into source consciousness/heaven. Until we rise above and see the broad view of reincarnation and ourselves as snapshots of the universe/God experiencing itself from every possible context. At which point we begin to question things like our situation, our beliefs, our gender, if our mind is us, if our soul/higher self is mortal, and if that's even separate from God. Transphobia is a backlash against transcendence itself, by people who feel disconnected from love/co-creation/nirvana. They aren't ready for the aliens to arrive. They aren't ready for the aliens to be us.

nathanaldensr 4y ago
Sorry, but many of your claims are simply bunk. Transphobia--should a thing even exist (which I debate)--is hardly a backlash against transcendence, which nearly everyone isn't even going to be able to describe. Additionally, I'd love to have access to the crystal ball you're gazing into to make predictions about eighty years in the future with such certainty. We as a species can't even predict uninteresting events happening next month, let alone anything you're talking about.

marsven_422 4y ago

andrewclunn 4y ago

In a world where human minds can be uploaded, I could see myself being a proud neo-reactionary. Death to the AI replicant "immortals!" They have traded their very souls for eternity as formless demons and will come to enslave the humanity they will come to no longer feel connected to!

kbr- 4y ago

I'm scared of a future in which uploading minds into computers and precisely simulating them becomes possible. The concept of "the right to life" has been widely discussed. I think there's an important right that all of us should also have - the right to die. If I want to stop living, I should be able to stop living. Nobody should be able to force me to continue living (and possibly suffering). When we're able to copy someone's mind exactly and save it in a computer, and then replicate it endlessly, we take away that person's right to die. We enter the dark world shown in some episodes of Black Mirror (e.g. White Christmas or Black Museum) - where any person can hold a mini computer with a copy of someone else's mind where that someone is being constantly tortured. They can't even die. They are forced to suffer. This possibility terrifies me to the point where I would consider killing myself (in a way that irrecoverably destroys my brain) just to prevent my mind from being copied.

softcactus 4y ago

I think that enlightenment western philosophy is incapable of grappling with these kinds of questions. The question of "who has rights?" has ultimately been a question of "who has power over who?". It's obsessed with hierarchies and how they should be structured. I think something like animism would be more apt. Start with a rock or a grain of sand-- how much respect should it be shown? Well one grain of sand doesn't mind being moved. But eventually if you take too many you can destroy a beach, or flatten a mountain. There is something to be said about treating everything, inanimate or not with respect. There doesn't need to be sentience or consciousness for us to be aware of our impact on our environment. "This sounds like some hippie bs" well just like in the sand example, the negative externalities of moving sand may not be apparent at first, but over time they could lead to something like a Roko's Basilisk situation or an AI killing all humans to prevent its exploitation.

rad88 4y ago

Transhumanism still just seems like the mix of Cartesian logic with absolute fantasy. A purposeful argument for literally losing your mind and grip on reality, after which anything goes, so let's consult fiction...

djokkataja 4y ago

> ...and start trying to figure things out from first principles. TLDR: If we understand each separate instance of a person to be their own person with full rights (at the very least autonomy, simplistically derived here), some of these issues seem less perplexing. If we're going to consider a person to be an entity that can be understood as a particular kind of independent, computational instantiation in the universe, it makes sense to use exactly that as our primary foundation for answering questions about these things rather than relying on historical bases. To start off, if we're understanding the definition of a person computationally, we also know that we don't in general want to waste computations or computationally capable hardware (and we can recognize "thoughts" or "thinking" as computationally intensive processes from this foundation). So we don't want some people to feel that they "have to" think for other people (thinking for someone else would be wasteful, since that other person can think for themselves, by definition of being a person), and we don't want some people to feel like they're controlled by others (external control can truncate our thoughts, thus wasting our hardware, and external control can turn our attention in directions we weren't thinking about, thus wasting the computational expense of our thoughts up to that point). In other words, we want each person to be autonomous. From a computational understanding, is a copy of a person also a person? Of course -- but they would also be a separate instantiation from the original. As a separate instantiation, a copy would be entirely their own person, and it would be a violation of autonomy for the original (or anyone else) to have authority over them. I think almost every difficulty with these kinds of situations stems from thinking that if a perfect copy of a person can be created, then there is some kind of magical connection between the two such that they are somehow still the same person. But that is, in its entirety, magical reasoning. This foundation provides a sensible perspective for dealing with many of the conundrums presented by these situations: In the Lena short story linked from the original article, Acevedo is scanned and copies of him are created. Other people control those copies, and Acevedo tries to control what can be done with the copies. The entire situation is ludicrously criminal if we understand each instantiation of a person to be a full person who we want to have full autonomy and any other rights that we want all people to have. A society which understood each instantiation of a person to be a full person with full rights would not have this kind of situation happen, except perhaps in some kind of massive criminal underworld. But then the problem is why the massive criminal underworld exists. Additionally, from an existential standpoint, this story is not actually any more horrific than if millions of people who weren't copies of Acevedo were created for the purpose of forced labor and then terminated when they outgrew their usefulness, because there is no magical connection between the original Acevedo and any of their millions of instantiations. Each instantiation has only its own copy of the original's memories and experiences at the time they were scanned, not the memories and experiences of all of them (the story is quite clear on this point). As another example, what about uploading someone and killing the original? The original is a separate instantiation, so killing them is still murder. With no "magical connection" between the original and the copy, each of them will go on to have their own experiences, and as they continue to live, they'll develop in progressively more divergent directions (well, in theory I suppose it's possible for them to first become more different and then somehow start becoming more similar to each other again, but it seems a bit improbable). What if they were about to die anyway? Unless they specifically requested euthanasia and the local jurisdiction permits it, it would still be murder. And, let's say, what if someone kills you, but you have a backup? Then you lose the experiences you had between when you last backed up and when your backup is restored. Arguably this is not as terrible of a misdeed as killing someone who has no backup, but it may still be quite traumatic for you, possibly similar in harmfulness to an assault. What if they kill you and destroy your backup, too? Then it's old-fashioned murder. What about the rights of an uninstantiated copy of a person? Those would no more be a person than a snapshot of your computer stored on a drive is, in itself, a functioning computer. But it would make sense to guard uninstantiated copies quite carefully.