7 ms·
For tech CEOs, the dystopia is the point
- nonrandomstring 2y ago> So it turns out it is aspirational branding, it’s just a deeply misanthropic variety Not misanthropic. Masochistic. I will to be writing something more substantial on this soon, but the gist is; a lot of these projects are knowingly self-destructive, and what their leaders crave is for others to join them in their expressins of self-harm.
- Terr_ 2y ago> Not misanthropic. Masochistic. I disagree: The real goal is to get others to endorse and support their worldview about how the world works and what their own place within it is. That doesn't come from the organizer enjoying pain/harm, but from desperately seeking to avoid experiencing a certain kind of it, emotional pain or harm to their ego. That said, if someone else is willing to harm themselves (physically, financially, legal jeopardy) to "prove" their allegiance to the narrative, that may be considered a more powerful and pleasurable endorsement. Also a sunk-cost also makes it harder for the person to change their mind later. In other words, more like "messianic cult-leader" and less "Pinhead from Hellraiser."
- nonrandomstring 2y agoYes, there's an element of messiah and saviour complex in there. > That doesn't come from the organizer enjoying pain/harm, but from desperately seeking to avoid experiencing a certain kind of it, emotional pain or harm to their ego. I think you actually describe a good picture of sado-masochism, the protective mechanism against emotions of unworthiness, though maybe you assumed I was emphasising the sadism side. Of course no actual tech leaders are exactly like that (they're real people with their own subtle issues that are personal, private and none of our business - even if they would not allow the same dignity for everyone else). We are talking about archetypes here. Jobs was paradigmatic. And not limited to tech, things like drugs (and the media/arts) are a space where you can achieve mass-reach much more easily. For me, the portrayal of Richard Sackler (Michael Stuhlbar) in Dopesick [0] is an even better depiction of the tech leader archetype than the fictional Peter Isherwood (Mark Rylance) in Don't Look Up [1]. We talk about "disruption" as if it were a synonym for progress but what I see is how many tech-obsessed "creator types" actually delight in destruction, and the pain experienced by others who they affect. Of course they would say, with that twisted Nietzschian faux-Christian 'compassion', that they "love" humanity and don't want to see anyone "left behind" (whether they want to be left out of an insane project or not). The recent Apple advert was a timely expression of this. I'm glad they made it. Its better to have this "out" than ignored. [0] https://en.wikipedia.org/wiki/Dopesick_(miniseries) https://en.wikipedia.org/wiki/Dopesick_(miniseries) [1] https://en.wikipedia.org/wiki/Don%27t_Look_Up https://en.wikipedia.org/wiki/Don%27t_Look_Up
- deleted 2y ago[deleted]
- Terr_ 2y ago> What’s the common denominator [...]? That the presumed user or owner of the product is the protagonist! [...] the narcissistic, us-against-the-collapsing-world mentality Another media tie-in here might be Atlas Shrugged. It too is a narrative to elevate an inventor--or more often investor--into an ubermensch hero role, one capable of projecting power against a dilemma that is mostly comprised of other people who failed to give the protagonist the respect/worship they were due. I don't think it's a cynical strategy to sell products per se, but it comes about because the person with influence over the marketing is able to pick and promote a flattering story or framing. ____________ Also, as referenced in the article, the classic: > Sci-Fi Author: "In my book I invented the Torment Nexus as a cautionary tale." > Tech Company: "At long last, we have created the Torment Nexus from the classic sci-fi novel Don't Create The Torment Nexus."
- brigadier132 2y ago> It too is a narrative to elevate an inventor--or more often investor--into an ubermensch hero role, one capable of projecting power against a dilemma that is mostly comprised of other people who failed to give the protagonist the respect/worship they were due I've never read atlas shrugged and don't intend on it. But responding to this specific comment, if this planet had 0 creators and only bureaucrats, critics and consumers it would quickly go to hell. Meanwhile, if we had no bureaucrats, critics, and consumers and only people that created things I'm fairly certain the world would be fine. > Sci-Fi Author: "In my book I invented the Torment Nexus as a cautionary tale." Just to be clear, we aren't talking about the torment nexus. We are talking about robots that can be interacted with using human language.
- s1artibartfast 2y agoTech CEOs and "tech culture" has a fundamentally different understanding of causality and human agency than the author. Cybertrucks arent what make the world of bladerunner a dystopia. AI isnt what makes the world of "Her" a dystopia. Techies see dystopian Sci-fi not as cautionary tale which suggests technology should be avoided, but rather a cautionary tale about how humans should interact with technology, and human behavior to be avoided. I think this can be argued to be an optimistic view of human capacity, choice, individual agency. Naturally, it conflicts with pessimistic views where humans have low-agency and are at the mercy of their environment. One side thinks that when presented unlimited ice cream, people can learn to control themselves. The other side thinks that when presented unlimited ice cream, people will eat themselves to death.
- 39896880 2y agoOne side has a basis in neuroscience[0], the other in a need to exploit naive optimism to make the stock go up. [0] https://www.penguinrandomhouse.com/books/592344/determined-by-robert-m-sapolsky/ https://www.penguinrandomhouse.com/books/592344/determined-b...
- s1artibartfast 2y agoThat is a very cynical take. Optimism and adaptation also have a basis in neuroscience. Neuroscience does not mean that everyone does the most degenerate thing possible at every opportunity. There are people that don't eat themselves to death when presented the option. There is also neuroscience behind discipline, self sacrifice, and every other positive virtue you can imagine.
- GauntletWizard 2y agoBoth sides are right, (mostly) about themselves.
- brigadier132 2y agoWell said. The author's argument is equivalent to saying advanced medical technology (or any advanced technology) is dystopian because it's in a bunch of scifi movies that happened to be dystopian.
- dnissley 2y agoWas I the only one who actually watched this movie? Her is not a dystopia. It's barely even cautionary. The main character seems better off having loved and lost than never having loved at all.
- deleted 2y ago[deleted]
- tivert 2y ago> Was I the only one who actually watched this movie? Her is not a dystopia. I watched it, and it kind of was a dystopia. Not in the classic jackboots and oppression sense, but one of pervasive social disconnection and isolation. I mean, the main character's job was literally to write "heartfelt" letters to other people's loved ones on their behalf, because they couldn't be bothered to do it themselves.
- lupire 2y agoThe dystopia in the movie was not the AI, though. The dystopia was what made the AI appealing to the protagonist.
- tschwimmer 2y agoI agree with you about it not being a dystopia but clearly it's a cautionary tale. The message I got from Her is not "Don't create the Torment Nexus" but "Touch Grass" or maybe "Don't force it." The narrator is profoundly socially alienated after his divorce. He's hung up on his ex-wife he's lonely, so he goes on all these dates with other people like Olivia Munn which are horribly awkward so he withdraws even further. Finally he finds Scarlett JohanssAIn and everything is perfect blah blah blah. But clearly it's not going to last as AI Scarlett clearly is in a different league and goes on to cheat on him and accelerate the singularity. He ends up with his awkward neighbor who isn't perfect and has her own skeletons but is more fundamentally compatible than other people he's been with. I actually just read the Wikipedia article about the writing of the script and Spike Jonze has explicitly said he was inspired by the false allure of chatting with a chatbot. So clearly the intent of the work is in strict tension with the GPT-4o demo. I'm honestly pretty puzzled by the tech CEOs constantly referencing fictional dystopias. I honestly cannot believe they're ignorant of the artistic intent of most of these works. I reread Snow Crash after the original Metaverse announcement and I kept thinking "Did he actually read this book?"
- karaterobot 2y agoI only saw Her once, when it came out, but I didn't view it as an AI dystopia. The dystopia existed before the AI, and was mostly meant to represent the world some of us already lived in, but which was not evenly distributed at the time. I thought the AI part was meant to be a metaphor for personal growth. I agree that trying to recreate the world of Snow Crash or Neuromancer is missing the point, and some people do seem to be trying to do that, but the central example used in this essay also feels like it's missing the point, just in a different way.
- roughly 2y agoHere's what gets me about this: Mark Zuckerberg bought like 1500 acres of land in Hawaii and is apparently building an underground apocalypse bunker on it. He's doing so, presumably, because he's worried about society collapsing. He's doing it with money that he's made from Facebook, which is the _reason_ society is collapsing. Mark Zuckerberg has worked his ass off for like 20 years to go live in an underground bunker. Granted, it's a large and luxurious underground bunker, but this does not seem like an improvement. Mark Zuckerberg controls the largest mass-persuasion device on earth - literally controls, he's got more than 50% of the shareholder votes - and is using the proceeds from owning the globe-spanning apocalypse machine that he's turned Facebook into to build himself a bunker underground to survive the apocalypse his machine is creating. Just turn off the goddamn machine and you can actually enjoy Hawaii, Mark. All the good stuff is above ground!
- toomuchtodo 2y agoThe mental model precludes the self awareness you describe. That’s why laws and regulators are needed: you cannot entirely take the lizard brain out of the human. You’re not going to convince people like Mark, but government can grind slowly, forever. No one else is going to protect you from these people at scale.
- roughly 2y agoOh I know that. The whole process is so obviously self-defeating that there’s clearly no actual rationality to it. It’s just a shame we’re all stuck on this boat with him.
- toomuchtodo 2y agoTrue story. Onward.
- SpicyLemonZest 2y agoWhat precisely do you mean when you say society is collapsing because of Facebook?
- gweinberg 2y agoYou don't learn reliable lessons from fiction. The author of "Don't create the Torment Nexus" may have intended it as a cautionary tale whose lesson is not to create the Torment Nexus, but that doesn't obligate the reader to come to that conclusion.Why shouldn't we say with a couple minor tweaks and a sexier name this "torment nexus" would be pretty cool? Conversely, most "utopias" sound like they would be pretty horrible to actually live in.
- brigadier132 2y agoYou only need to turn the dial a bit to get the pleasure nexus and we'd all be happy.
- smegsicle 2y ago> most "utopias" sound like they would be pretty horrible to actually live in. are there counterexamples? i thought the only difference between utopia and dystopia would be whether the speaker is promoting it or not
- n4r9 2y agoThere are plenty of examples of "critical" utopias which do not try to be absolutely perfect, and acknowledge where they fall down. Anarres in the Dispossessed by Le Guin, and the Culture novels by Iain M Banks might be two examples. They're still vastly preferable (imo) to current life on earth.
- gweinberg 2y agoHuxley's _Island_ didn't seem too bad. I think the general problem is, if you want to completely eliminate crime, poverty, and conflict in general, you have to have a society that is extremely regimented, and you have to be willing to kill off anyone who seems "defective".
- tivert 2y ago> The author of "Don't create the Torment Nexus" may have intended it as a cautionary tale whose lesson is not to create the Torment Nexus, but that doesn't obligate the reader to come to that conclusion. One of the points of the Torment Nexus joke is that the Tech Company is totally oblivious to the fact that "Don't create the Torment Nexus" was describing something bad. > Why shouldn't we say with a couple minor tweaks and a sexier name this "torment nexus" would be pretty cool? I actually think your defense there actually gets to the core error: things can be simultaneously cool and awful. For instance: nuclear war. On the one hand, you have high tech rockets buzzing around insanely fast while precisely dropping incredibly powerful fireworks; which is admittedly pretty "cool," on the other hand, you have mass death and suffering. The Tech Company people are shallowly responding to just the "cool" part, and missing all the rest.
- gary_0 2y agoWell, just because he made iPads inspired by Star Trek doesn't mean Steve Jobs wanted war with the Klingons. Although apparently his company wants musical instruments and works of art to be crushed out of existence and replaced with a cold digital slab. So regardless of the level of malice, it's obvious that the tech industry is getting creepily out of touch with non-virtual reality.
- mondrian 2y agoI think the simpler explanation is that they’re struck by the style of these artworks, and aren’t interested in referencing the cautionary tale aspect. Bladerunner, Snowcrash, The Matrix, feature ‘cool’ characters and settings. It’s cool to imagine being a crew member on the Nebuchadnezzar or whatever. No one references 1984, Brazil, or Children of Men to advertise their product, because those artworks create an uninviting, flat bleakness. Maybe some blame should go toward authors who work hard to draw you in with style and vibe, in order to tell you a cautionary tale. Many in the audience probably miss the tale entirely, and are just immersed in the looks and vibe.
- pixl97 2y agoAt long last, we have created the Torment Nexus from classic sci-fi novel Don't Create The Torment Nexus because they had really cool looking uniforms.
- hi-v-rocknroll 2y agoNit: "Artwork" is singulare tantum and so "artworks" isn't a word.
- BobaFloutist 2y ago"Arts work"
- lupire 2y ago"singulare" and "tantum" are not English words, but "artworks" is.
- CRConrad 2y agoPretty much every possible word is an English word, it seems. "English is a whore, sleeping with every language she encounters, and with a very successful sideline in pickpocketing, rifling through all their pockets for any word she can find", or however the old quote goes.
- deleted 2y ago[deleted]
- nunez 2y agoWe saw dystopia; they saw unbelievable profit Also, to be fair: - Siri had come out two years before "Her" was released to theaters, so the critique was more about voice assistants (which is a form of AI) than AGI or the AI sota. - The closest thing to "Her" would be a Humane Pin with 4o or higher. A Humane Pin like experience that didn't suck would honestly be super cool. That would allow me to leave the phone at home!
- CRConrad 2y ago> AGI or the AI sota. Soma? Soda? Sofa...?