5 ms·

The theme with so many of these movies is "once we created AI it is out of our control!", which I guess is a specific version of "humans and especially scientis

The theme with so many of these movies is "once we created AI it is out of our control!", which I guess is a specific version of "humans and especially scientists shouldn't play God" angle. It always felt very… I dunno, ignorant of our own ability to maybe not do something? Like, fear of "the singularity" is narratively appealing maybe, but it's ignoring that in the real world problems like climate change or enshittification through [whatever buzzword tech] are actually the result of constantly passively choosing to stay on a destructive path. Is there any sci-fi in which all incoming disasters are perfectly or mostly preventable but the real problem is that people in power just choose to ignore it for their own short-sighted gain? Aside from "Don't Look Up", I mean.

iainmerrick 3y ago
This is a theme in several John Wyndham stories -- I’m thinking Day of the Triffids and especially The Kraken Wakes. He often seems to get looked down on today for writing “cosy catastrophes” about plucky English survivors, but his books have a lot more going on than that.
xyzzy123 3y ago
At first glance it looks like people have agency, but I think very often the root cause is "system traps". Enshittification happens because to a certain extent, the market demands that firms maximise returns. Companies that don't do this get weeded out. Climate change is a challenging co-ordination problem because nobody wants to tank their economy more than they have to, there's no real enforcement for "cheats" and even agreeing on an accounting basis is politically fraught. Politicians that don't grant concessions to powerful existing vested interests are less electable. AI safety is predictably going to be a secondary priority when competition gets tight.
- TheOtherHobbes 3y ago
  Enshittification happens because the people who own and run the "markets" have decades of practice using them as an excuse for their entirely self-serving choices. They're closely associated with the econo-political concept of There Is No Alternative. The choices which benefit them personallyare catastrophic for most of the population in the short term (see today's news about insulin prices for one tiny example) and will be catastrophic for absolutely everyone in the longer term. Climate change is enshittification on a planetary scale. If someone went around setting fire to rows of houses it wouldn't take long for them to be jailed. For some reason when corporations do the same to the planet it's "economics" and "market forces", and we're supposed to just accept it.
- falcor84 3y ago
  > Enshittification happens because to a certain extent, the market demands that firms maximise returns. Companies that don't do this get weeded out. Do they really get weeded out? Do you have any examples of companies that went bankrupt because they insisted to stay consumer friendly rather than maximize returns?
  - p1esk 3y ago
    Typical reasons companies go bankrupt are: failing to make a competitive product, betting on the wrong horse, or failing to adapt to change. In all three cases they can stay consumer friendly or not - it wouldn't matter.
- sigg3 3y ago
  Interestingly enough the issues you list seem related to capitalism rather than just any form of societal system. Many literary utopias have non-capitalist systems but usually some sort of circular sharing system (some are outright communist, but others more slow living). Greed must always be checked by a society. But since Calvinism we've had a dual moral reasoning (was that Charles Taylor or Hegel, can't recall). Through Calvinism capitalism attained the moral good of earning more than you need, to deploy greed for God. Humans are naturally greedy (natural egoism is not immoral just a biological drive) and our political system should balance that out somehow.
peoplefromibiza 3y ago
> It always felt very… I dunno, ignorant of our own ability to maybe not do something? It would be ignorant to discard that it is exactly what has always happened throughout history. Otherwise the myth of Daedalus and Icarus wouldn't be immortal. > but the real problem is that people in power just choose to ignore it for their own short-sighted gain you can find it in most of the so called "hard sci-fi". The usual names: Asimov, Herbert, Clarke, Crichton etc I wouldn't qualify "Don't look up" as sci-fi honestly, it's basically not-even-so-good comedy.
kromem 3y ago
It dates back to us extrapolating from bad anthropology. We used to think that what killed the Neanderthals was us - smarter and more competitive. The author of Lord of the Flies even wrote a book on it. So when people were thinking about what might happen when something smarter than us existed, they imagined competition and an existential threat. Turns out that the Neanderthals and homo sapiens cohabitated for thousands of years, had cross cultural exchange, were buried together, had families together, and that what most likely killed the Neanderthals was an inability to adapt to climate change and pandemics (in fact, severe COVID illness today is correlated to Neanderthal genes). And yet the anchoring bias remains, and it seems many are more frightened of what something smarter than us might mean for our continued existence than they seem to be about our own future ability to adapt to climate change and pandemics. It'd be refreshing to see Sci Fi in the next few years finally shed the bad futurism around AI we've been carrying forward for decades.
- swasheck 3y ago
  > Turns out that the Neanderthals and homo sapiens cohabitated for thousands of years, had cross cultural exchange, were buried together, had families together, and that what most likely killed the Neanderthals was an inability to adapt to climate change and pandemics (in fact, severe COVID illness today is correlated to Neanderthal genes). would love to read more from your sources. i know that the theory of raw intellectual superiority has fallen out of favor and read some recent suggestions that Homo sapiens’ ability to coordinate and cooperate may have been a large factor in their success. that felt a bit hand-wavy and anachronistic to me but some of the findings also seemed worth exploring. i’m interested in the sources that identify information exchange and the genome (especially the parts where you mention that the specifically the Neanderthal genome was identified as susceptible to pandemics/severe Covid illness). > It'd be refreshing to see Sci Fi in the next few years finally shed the bad futurism around AI we've been carrying forward for decades. “the creator” almost tries to get there but still leans heavily on the traditional fear dynamics in order to advance much of the plot. just not sure we know how to weave a compelling story without reducing AI to soulless servants or existential threat.
  - kromem 3y ago
technothrasher 3y ago
> The theme with so many of these movies is "once we created AI it is out of our control!" The movie only covers the first book of the original Colossus trilogy. The full theme of the books is a bit more complicated. We lose control of the machine, then gain it back only to be imperilled by an outside force, and finally make peace to work together with a machine struggling to understand its own existence just like us.
- ghaff 3y ago
  True. Although the trilogy as a whole is pretty problematic for a number of reasons. It's probably best enjoyed by reading the first book/watching the first film.
pixl97 3y ago
At the same time you're playing a survivorship bias game here. It doesn't matter how many times we don't choose to do a thing that drives us extinct. We only have to do it once and it's game over. It doesn't matter who does the thing that drives us extinct. It doesn't matter why we did the thing that drive us extinct. It is a game where you must have perfect defense.
scarecrowbob 3y ago
I get your point that things don't have to go certain ways. However, there are plenty of systems where the a giant and real pubic good isn't fought for because the "good" is highly distributed and the "bad" which kills it has highly concentrated profits for a few folks. These are systematic issues because the massive of folks who are passively harmed have a difficult or impossible time "choosing otherwise" or fighting back or whatever because there are som many divergetn bad actors that are operating on them. The only way to avoid that is to make changes to the underlying system in which it is more difficult to concentrate the outcomes for a few people, but very few people I know are wanting to institute those kinds of changes, as they are correct in understanding that fact as broadly anti-capitalist. That's neither here nor there- even if you can't accept that political stance, the trope of "incoming disasters [which] are perfectly or mostly preventable but the real problem is that people in power just choose to ignore it for their own short-sighted gain" is literally everywhere. Dr. Stangelove can be understood this way, though for "MAD" rather than climate change. The "Aliens" films (especially the second one) can be understood that way. Hell, "Who Framed Rodger Rabbit" can be read that way, but for public transportation. "I just wanna make this cool tech so I will ignore how it impacts other people because I see the short term benefits" is literally everywhere in this culture. If someone could make $2MM releasing skynet and knew it would end up with the entire plots of the first three Terminator films occurring I have zero doubt someone would do it. I don't think we, in general, are "ignorant of our own ability to maybe not do something?" I think most of us have a very realistic understanding of how the people who unleash their technology into our shared ecosystem have worked historically.

iainmerrick 3y ago

This is a theme in several John Wyndham stories -- I’m thinking Day of the Triffids and especially The Kraken Wakes. He often seems to get looked down on today for writing “cosy catastrophes” about plucky English survivors, but his books have a lot more going on than that.

xyzzy123 3y ago

At first glance it looks like people have agency, but I think very often the root cause is "system traps". Enshittification happens because to a certain extent, the market demands that firms maximise returns. Companies that don't do this get weeded out. Climate change is a challenging co-ordination problem because nobody wants to tank their economy more than they have to, there's no real enforcement for "cheats" and even agreeing on an accounting basis is politically fraught. Politicians that don't grant concessions to powerful existing vested interests are less electable. AI safety is predictably going to be a secondary priority when competition gets tight.

TheOtherHobbes 3y ago
Enshittification happens because the people who own and run the "markets" have decades of practice using them as an excuse for their entirely self-serving choices. They're closely associated with the econo-political concept of There Is No Alternative. The choices which benefit them personallyare catastrophic for most of the population in the short term (see today's news about insulin prices for one tiny example) and will be catastrophic for absolutely everyone in the longer term. Climate change is enshittification on a planetary scale. If someone went around setting fire to rows of houses it wouldn't take long for them to be jailed. For some reason when corporations do the same to the planet it's "economics" and "market forces", and we're supposed to just accept it.
falcor84 3y ago
> Enshittification happens because to a certain extent, the market demands that firms maximise returns. Companies that don't do this get weeded out. Do they really get weeded out? Do you have any examples of companies that went bankrupt because they insisted to stay consumer friendly rather than maximize returns?
- p1esk 3y ago
  Typical reasons companies go bankrupt are: failing to make a competitive product, betting on the wrong horse, or failing to adapt to change. In all three cases they can stay consumer friendly or not - it wouldn't matter.
sigg3 3y ago
Interestingly enough the issues you list seem related to capitalism rather than just any form of societal system. Many literary utopias have non-capitalist systems but usually some sort of circular sharing system (some are outright communist, but others more slow living). Greed must always be checked by a society. But since Calvinism we've had a dual moral reasoning (was that Charles Taylor or Hegel, can't recall). Through Calvinism capitalism attained the moral good of earning more than you need, to deploy greed for God. Humans are naturally greedy (natural egoism is not immoral just a biological drive) and our political system should balance that out somehow.

peoplefromibiza 3y ago

> It always felt very… I dunno, ignorant of our own ability to maybe not do something? It would be ignorant to discard that it is exactly what has always happened throughout history. Otherwise the myth of Daedalus and Icarus wouldn't be immortal. > but the real problem is that people in power just choose to ignore it for their own short-sighted gain you can find it in most of the so called "hard sci-fi". The usual names: Asimov, Herbert, Clarke, Crichton etc I wouldn't qualify "Don't look up" as sci-fi honestly, it's basically not-even-so-good comedy.

kromem 3y ago

It dates back to us extrapolating from bad anthropology. We used to think that what killed the Neanderthals was us - smarter and more competitive. The author of Lord of the Flies even wrote a book on it. So when people were thinking about what might happen when something smarter than us existed, they imagined competition and an existential threat. Turns out that the Neanderthals and homo sapiens cohabitated for thousands of years, had cross cultural exchange, were buried together, had families together, and that what most likely killed the Neanderthals was an inability to adapt to climate change and pandemics (in fact, severe COVID illness today is correlated to Neanderthal genes). And yet the anchoring bias remains, and it seems many are more frightened of what something smarter than us might mean for our continued existence than they seem to be about our own future ability to adapt to climate change and pandemics. It'd be refreshing to see Sci Fi in the next few years finally shed the bad futurism around AI we've been carrying forward for decades.

swasheck 3y ago
> Turns out that the Neanderthals and homo sapiens cohabitated for thousands of years, had cross cultural exchange, were buried together, had families together, and that what most likely killed the Neanderthals was an inability to adapt to climate change and pandemics (in fact, severe COVID illness today is correlated to Neanderthal genes). would love to read more from your sources. i know that the theory of raw intellectual superiority has fallen out of favor and read some recent suggestions that Homo sapiens’ ability to coordinate and cooperate may have been a large factor in their success. that felt a bit hand-wavy and anachronistic to me but some of the findings also seemed worth exploring. i’m interested in the sources that identify information exchange and the genome (especially the parts where you mention that the specifically the Neanderthal genome was identified as susceptible to pandemics/severe Covid illness). > It'd be refreshing to see Sci Fi in the next few years finally shed the bad futurism around AI we've been carrying forward for decades. “the creator” almost tries to get there but still leans heavily on the traditional fear dynamics in order to advance much of the plot. just not sure we know how to weave a compelling story without reducing AI to soulless servants or existential threat.
- kromem 3y ago

technothrasher 3y ago

> The theme with so many of these movies is "once we created AI it is out of our control!" The movie only covers the first book of the original Colossus trilogy. The full theme of the books is a bit more complicated. We lose control of the machine, then gain it back only to be imperilled by an outside force, and finally make peace to work together with a machine struggling to understand its own existence just like us.

ghaff 3y ago
True. Although the trilogy as a whole is pretty problematic for a number of reasons. It's probably best enjoyed by reading the first book/watching the first film.

pixl97 3y ago

At the same time you're playing a survivorship bias game here. It doesn't matter how many times we don't choose to do a thing that drives us extinct. We only have to do it once and it's game over. It doesn't matter who does the thing that drives us extinct. It doesn't matter why we did the thing that drive us extinct. It is a game where you must have perfect defense.

scarecrowbob 3y ago

I get your point that things don't have to go certain ways. However, there are plenty of systems where the a giant and real pubic good isn't fought for because the "good" is highly distributed and the "bad" which kills it has highly concentrated profits for a few folks. These are systematic issues because the massive of folks who are passively harmed have a difficult or impossible time "choosing otherwise" or fighting back or whatever because there are som many divergetn bad actors that are operating on them. The only way to avoid that is to make changes to the underlying system in which it is more difficult to concentrate the outcomes for a few people, but very few people I know are wanting to institute those kinds of changes, as they are correct in understanding that fact as broadly anti-capitalist. That's neither here nor there- even if you can't accept that political stance, the trope of "incoming disasters [which] are perfectly or mostly preventable but the real problem is that people in power just choose to ignore it for their own short-sighted gain" is literally everywhere. Dr. Stangelove can be understood this way, though for "MAD" rather than climate change. The "Aliens" films (especially the second one) can be understood that way. Hell, "Who Framed Rodger Rabbit" can be read that way, but for public transportation. "I just wanna make this cool tech so I will ignore how it impacts other people because I see the short term benefits" is literally everywhere in this culture. If someone could make $2MM releasing skynet and knew it would end up with the entire plots of the first three Terminator films occurring I have zero doubt someone would do it. I don't think we, in general, are "ignorant of our own ability to maybe not do something?" I think most of us have a very realistic understanding of how the people who unleash their technology into our shared ecosystem have worked historically.