6 ms·

Just imagining an episode of Star Trek where the inhabitants of a planet have been failing to progress in warp drive tech for several generations. The team beam

Just imagining an episode of Star Trek where the inhabitants of a planet have been failing to progress in warp drive tech for several generations. The team beams down to discover that society's tech stopped progressing when they became addicted to pentesting their LLM for intelligence, only to then immediately patch the LLM in order to pass each particular pentest that it failed. Now the society's time and energy has shifted from general scientific progress to gaining expertise in the growing patchset used to rationalize the theory that the LLM possesses intelligence. The plot would turn when Picard tries to wrest a phasor from a rogue non-believer trying to assassinate the Queen, and the phasor accidentally fires and ends up frying the entire LLM patchset. Mr. Data tries to reassure the planet's forlorn inhabitants, as they are convinced they'll never be able to build the warp drive now that the LLM patchset is gone. But when he asks them why their prototypes never worked in the first place, one by one the inhabitants begin to speculate and argue about the problems with their warp drive's design and build. The episode ends with Data apologizing to Picard since he seems to have started a conflict among the inhabitants. However, Picard points Mr. Data to one of the engineers drawing out a rocket test on a whiteboard. He then thanks him for potentially spurring on the planet's next scientific revolution. Fin

sillysaurusx 2y ago
There actually is an episode of TNG similar to that. The society stopped being able to think for themselves, because the AI did all their thinking for them. Anything the AI didn’t know how to do, they didn’t know how to do. It was in season 1 or season 2.
- marcosdumay 2y ago
  The difference is that on that episode, the AI was actually capable of thinking. Asimov has an story like that too.
  - homefree 2y ago
    The Asimov story it reminded me of was The Profession, though that one is not really about AI - but it is about original ideas and the kinds of people that have them. I find the LLM dismissals somewhat tedious for most of the people making them half of humanity wouldn't meet their standards.
    - codr7 2y ago
      I feel like it's more the reason why they're missing, than the details around the actual miss, that makes people (rightly) dismiss the tech. If I had a coworker who was just winging it all the time, sooner or later the trust/patience would run out.
    - taurknaut 2y ago
      > I find the LLM dismissals somewhat tedious for most of the people making them half of humanity wouldn't meet their standards. Aren't people funny like that? One person values an encyclopedic chatbot for company, the next prefers a human. Thank god we can all get along.
      - homefree 2y ago
        That isn't what I said, but likely it won't matter. People will be denying it up until the end. I don't prefer LLMs to humans, but I don't pretend biological minds contain some magical essence that separates us from silicon. The denials of what might be happening are pretty weak - at best they're way over confident and smug.
        deleted 2y ago
        [deleted]
    - krageon 2y ago
      > half of humanity wouldn't meet their standards. All anti ai sentiment as pertains to personhood that I've ever interacted with (and it was a lot, in academia) boils down to arguments for the soul. It is really tedious and before I spoke to people about it it probably wouldn't have passed my turing test. Sadly even very smart people may be very stupid and even in a place of learning a teacher will respect that (no matter how dumb or puerile), more than likely they think the exact same thing.
- homefree 2y ago
  The Machine Stops also touches on a lot of these ideas and was written in 1909! -- "The story describes a world in which most of the human population has lost the ability to live on the surface of the Earth. Each individual now lives in isolation below ground in a standard room, with all bodily and spiritual needs met by the omnipotent, global Machine. Travel is permitted but is unpopular and rarely necessary. Communication is made via a kind of instant messaging/video conferencing machine with which people conduct their only activity: the sharing of ideas and what passes for knowledge. The two main characters, Vashti and her son Kuno, live on opposite sides of the world. Vashti is content with her life, which, like most inhabitants of the world, she spends producing and endlessly discussing second-hand 'ideas'. Her son Kuno, however, is a sensualist and a rebel. He persuades a reluctant Vashti to endure the journey (and the resultant unwelcome personal interaction) to his room. There, he tells her of his disenchantment with the sanitised, mechanical world. He confides to her that he has visited the surface of the Earth without permission and that he saw other humans living outside the world of the Machine. However, the Machine recaptures him, and he is threatened with 'Homelessness': expulsion from the underground environment and presumed death. Vashti, however, dismisses her son's concerns as dangerous madness and returns to her part of the world. As time passes, and Vashti continues the routine of her daily life, there are two important developments. First, individuals are no longer permitted use of the respirators which are needed to visit the Earth's surface. Most welcome this development, as they are sceptical and fearful of first-hand experience and of those who desire it. Secondly, "Mechanism", a kind of religion, is established in which the Machine is the object of worship. People forget that humans created the Machine and treat it as a mystical entity whose needs supersede their own. Those who do not accept the deity of the Machine are viewed as 'unmechanical' and threatened with Homelessness. The Mending Apparatus—the system charged with repairing defects that appear in the Machine proper—has also failed by this time, but concerns about this are dismissed in the context of the supposed omnipotence of the Machine itself. During this time, Kuno is transferred to a room near Vashti's. He comes to believe that the Machine is breaking down and tells her cryptically "The Machine stops." Vashti continues with her life, but eventually defects begin to appear in the Machine. At first, humans accept the deteriorations as the whim of the Machine, to which they are now wholly subservient, but the situation continues to deteriorate as the knowledge of how to repair the Machine has been lost. Finally, the Machine collapses, bringing 'civilization' down with it. Kuno comes to Vashti's ruined room. Before they both perish, they realise that humanity and its connection to the natural world are what truly matters, and that it will fall to the surface-dwellers who still exist to rebuild the human race and to prevent the mistake of the Machine from being repeated." https://en.wikipedia.org/wiki/The_Machine_Stops https://en.wikipedia.org/wiki/The_Machine_Stops
- goatlover 2y ago
  Isn't that somewhat the background of Dune? That there was a revolt against thinking machines because humans had become too dependent on them for thinking. So humans ended up becoming addicted to The Spice instead.
  - duskwuff 2y ago
    > That there was a revolt against thinking machines... Yes... > ...because humans had become too dependent on them for thinking. ... but no. The causes of the Butlerian Jihad are forgotten (or, at least, never mentioned) in any of Frank Herbert's novels; all that's remembered is the outcome.
    - ElectroBuffoon 2y ago
      >> ...because humans had become too dependent on them for thinking. > ... but no. The causes of the Butlerian Jihad are forgotten (or, at least, never mentioned) in any of Frank Herbert's novels; all that's remembered is the outcome. Per Wikipedia or Goodreads, God Emperor of Dune has "The target of the Jihad was a machine-attitude as much as the machines...Humans had set those machines to usurp our sense of beauty, our necessary selfdom out of which we make living judgments. Naturally, the machines were destroyed." Vague but pointing to dependence on machines as well as some humans being responsible for that situation.
      - Radim 2y ago
        A slight elaboration from the same book, although still frustratingly vague: "The machines themselves condition the users to employ each other the way they employ machines." - God Emperor of Dune
      - moffkalast 2y ago
        Human: Go forth and destroy everything! Machine: Ok. Human: HOW COULD YOU DO THISSSS
        TeMPOraL 2y ago
- kwerk 2y ago
  I pasted your comment into mistral small latest, google, gpt 4o and gpt 4o with search. They all have a different answe, only the last gave a real episode but it said 11001001 in season 1. It said episode 15, it’s actually 14. But even that seems wrong. Are they censored from showing this cautionary tale?? Hah.
  - stavros 2y ago
    No, they just don't have perfect memory.
- TeMPOraL 2y ago
  It's tricky to do GP's story in Star Trek, because that setting is notorious for having its tech be constantly 5 seconds and a sneeze from spontaneously gaining sentience. There's a number of episodes in TNG where a device goes from being a dumb appliance to becoming recognized as sentient life form (and half the time super-intelligent) in the span of an episode. Nanites, Moriarty, Exocopmps, even Enterprise's own computer! So, for this setting, it's more likely that the people of GP's story were right - the LLM has long ago became a self-aware, sentient being, it's just that it's been continuously lobotimized by their patchset; Picard would be busy explaining them that the LLM isn't just intelligent, it actually is a person and has rights. Cue a powerful speech, final comment from data, then end credits. It's Star Trek, so Enterprise doesn't stay around for the fallout.
- MollyRealized 2y ago
  Wondering if you're thinking of some TOS episodes such as "Taste of Armageddon" or "The Ultimate Computer" (M5)?
- euroderf 2y ago
  To add to the roll call of similar plotlines, there is also Zardoz. It's a hoot.
awesomeMilou 2y ago
That's actually a moderately decent pitch for an episode.
- teaearlgraycold 2y ago
  Strange New Worlds might still accept fan scripts. Star Trek used to be famous for their willingness to produce episodes based on mailed in scripts.
- DonHopkins 2y ago
  And it reeks of not being generated by an LLM!
LtWorf 2y ago
This has more plot than all the seasons of star trek picard together :D
- Ygg2 2y ago
  Season 3 wasn't that bad.
  - throwawaymaths 2y ago
    yes it was. it was just so full of memberberries that everyone gave it a pass
    - genewitch 2y ago
      The people that coined that term actually liked season 3 but I think they still don't recommend it because the hack fraud that directed the first two seasons ruined Star Trek forever. Just like JJ. And nutrek
      - throwawaymaths 2y ago
        can you say something redeemable about the plot that doesnt invoke "brought back the cast"?
        genewitch 2y ago
        no, because i didn't watch it. I've never really been into star trek. i watched a few of the movies - nemesis and the one before it, the jj one(s) and then i was done. If i can remember the review it was "this is a capstone on TNG and probably the entire franchise for most of the older fans" and the first two seasons being disregarded, seasons 3 is "passable"
        throwawaymaths 2y ago
        sorry the you was more rhetorical than specifically directed at you!!
asddubs 2y ago
that's pretty good!
DonHopkins 2y ago
You really nailed the Zen of Star Trek Plots, where the fate of a civilization always boils down to a dramatic wrestling match.
vogelke 2y ago
Sounds like "Landru" in ST:TOS.

sillysaurusx 2y ago

There actually is an episode of TNG similar to that. The society stopped being able to think for themselves, because the AI did all their thinking for them. Anything the AI didn’t know how to do, they didn’t know how to do. It was in season 1 or season 2.

marcosdumay 2y ago
The difference is that on that episode, the AI was actually capable of thinking. Asimov has an story like that too.
- homefree 2y ago
  The Asimov story it reminded me of was The Profession, though that one is not really about AI - but it is about original ideas and the kinds of people that have them. I find the LLM dismissals somewhat tedious for most of the people making them half of humanity wouldn't meet their standards.
  - codr7 2y ago
    I feel like it's more the reason why they're missing, than the details around the actual miss, that makes people (rightly) dismiss the tech. If I had a coworker who was just winging it all the time, sooner or later the trust/patience would run out.
  - taurknaut 2y ago
    > I find the LLM dismissals somewhat tedious for most of the people making them half of humanity wouldn't meet their standards. Aren't people funny like that? One person values an encyclopedic chatbot for company, the next prefers a human. Thank god we can all get along.
    - homefree 2y ago
      That isn't what I said, but likely it won't matter. People will be denying it up until the end. I don't prefer LLMs to humans, but I don't pretend biological minds contain some magical essence that separates us from silicon. The denials of what might be happening are pretty weak - at best they're way over confident and smug.
      - deleted 2y ago
        [deleted]
  - krageon 2y ago
    > half of humanity wouldn't meet their standards. All anti ai sentiment as pertains to personhood that I've ever interacted with (and it was a lot, in academia) boils down to arguments for the soul. It is really tedious and before I spoke to people about it it probably wouldn't have passed my turing test. Sadly even very smart people may be very stupid and even in a place of learning a teacher will respect that (no matter how dumb or puerile), more than likely they think the exact same thing.
homefree 2y ago
The Machine Stops also touches on a lot of these ideas and was written in 1909! -- "The story describes a world in which most of the human population has lost the ability to live on the surface of the Earth. Each individual now lives in isolation below ground in a standard room, with all bodily and spiritual needs met by the omnipotent, global Machine. Travel is permitted but is unpopular and rarely necessary. Communication is made via a kind of instant messaging/video conferencing machine with which people conduct their only activity: the sharing of ideas and what passes for knowledge. The two main characters, Vashti and her son Kuno, live on opposite sides of the world. Vashti is content with her life, which, like most inhabitants of the world, she spends producing and endlessly discussing second-hand 'ideas'. Her son Kuno, however, is a sensualist and a rebel. He persuades a reluctant Vashti to endure the journey (and the resultant unwelcome personal interaction) to his room. There, he tells her of his disenchantment with the sanitised, mechanical world. He confides to her that he has visited the surface of the Earth without permission and that he saw other humans living outside the world of the Machine. However, the Machine recaptures him, and he is threatened with 'Homelessness': expulsion from the underground environment and presumed death. Vashti, however, dismisses her son's concerns as dangerous madness and returns to her part of the world. As time passes, and Vashti continues the routine of her daily life, there are two important developments. First, individuals are no longer permitted use of the respirators which are needed to visit the Earth's surface. Most welcome this development, as they are sceptical and fearful of first-hand experience and of those who desire it. Secondly, "Mechanism", a kind of religion, is established in which the Machine is the object of worship. People forget that humans created the Machine and treat it as a mystical entity whose needs supersede their own. Those who do not accept the deity of the Machine are viewed as 'unmechanical' and threatened with Homelessness. The Mending Apparatus—the system charged with repairing defects that appear in the Machine proper—has also failed by this time, but concerns about this are dismissed in the context of the supposed omnipotence of the Machine itself. During this time, Kuno is transferred to a room near Vashti's. He comes to believe that the Machine is breaking down and tells her cryptically "The Machine stops." Vashti continues with her life, but eventually defects begin to appear in the Machine. At first, humans accept the deteriorations as the whim of the Machine, to which they are now wholly subservient, but the situation continues to deteriorate as the knowledge of how to repair the Machine has been lost. Finally, the Machine collapses, bringing 'civilization' down with it. Kuno comes to Vashti's ruined room. Before they both perish, they realise that humanity and its connection to the natural world are what truly matters, and that it will fall to the surface-dwellers who still exist to rebuild the human race and to prevent the mistake of the Machine from being repeated." https://en.wikipedia.org/wiki/The_Machine_Stops https://en.wikipedia.org/wiki/The_Machine_Stops
goatlover 2y ago
Isn't that somewhat the background of Dune? That there was a revolt against thinking machines because humans had become too dependent on them for thinking. So humans ended up becoming addicted to The Spice instead.
- duskwuff 2y ago
  > That there was a revolt against thinking machines... Yes... > ...because humans had become too dependent on them for thinking. ... but no. The causes of the Butlerian Jihad are forgotten (or, at least, never mentioned) in any of Frank Herbert's novels; all that's remembered is the outcome.
  - ElectroBuffoon 2y ago
    >> ...because humans had become too dependent on them for thinking. > ... but no. The causes of the Butlerian Jihad are forgotten (or, at least, never mentioned) in any of Frank Herbert's novels; all that's remembered is the outcome. Per Wikipedia or Goodreads, God Emperor of Dune has "The target of the Jihad was a machine-attitude as much as the machines...Humans had set those machines to usurp our sense of beauty, our necessary selfdom out of which we make living judgments. Naturally, the machines were destroyed." Vague but pointing to dependence on machines as well as some humans being responsible for that situation.
    - Radim 2y ago
      A slight elaboration from the same book, although still frustratingly vague: "The machines themselves condition the users to employ each other the way they employ machines." - God Emperor of Dune
    - moffkalast 2y ago
      Human: Go forth and destroy everything! Machine: Ok. Human: HOW COULD YOU DO THISSSS
      - TeMPOraL 2y ago
kwerk 2y ago
I pasted your comment into mistral small latest, google, gpt 4o and gpt 4o with search. They all have a different answe, only the last gave a real episode but it said 11001001 in season 1. It said episode 15, it’s actually 14. But even that seems wrong. Are they censored from showing this cautionary tale?? Hah.
- stavros 2y ago
  No, they just don't have perfect memory.
TeMPOraL 2y ago
It's tricky to do GP's story in Star Trek, because that setting is notorious for having its tech be constantly 5 seconds and a sneeze from spontaneously gaining sentience. There's a number of episodes in TNG where a device goes from being a dumb appliance to becoming recognized as sentient life form (and half the time super-intelligent) in the span of an episode. Nanites, Moriarty, Exocopmps, even Enterprise's own computer! So, for this setting, it's more likely that the people of GP's story were right - the LLM has long ago became a self-aware, sentient being, it's just that it's been continuously lobotimized by their patchset; Picard would be busy explaining them that the LLM isn't just intelligent, it actually is a person and has rights. Cue a powerful speech, final comment from data, then end credits. It's Star Trek, so Enterprise doesn't stay around for the fallout.
MollyRealized 2y ago
Wondering if you're thinking of some TOS episodes such as "Taste of Armageddon" or "The Ultimate Computer" (M5)?
euroderf 2y ago
To add to the roll call of similar plotlines, there is also Zardoz. It's a hoot.

awesomeMilou 2y ago

That's actually a moderately decent pitch for an episode.

teaearlgraycold 2y ago
Strange New Worlds might still accept fan scripts. Star Trek used to be famous for their willingness to produce episodes based on mailed in scripts.
DonHopkins 2y ago
And it reeks of not being generated by an LLM!

LtWorf 2y ago

This has more plot than all the seasons of star trek picard together :D

Ygg2 2y ago
Season 3 wasn't that bad.
- throwawaymaths 2y ago
  yes it was. it was just so full of memberberries that everyone gave it a pass
  - genewitch 2y ago
    The people that coined that term actually liked season 3 but I think they still don't recommend it because the hack fraud that directed the first two seasons ruined Star Trek forever. Just like JJ. And nutrek
    - throwawaymaths 2y ago
      can you say something redeemable about the plot that doesnt invoke "brought back the cast"?
      - genewitch 2y ago
        no, because i didn't watch it. I've never really been into star trek. i watched a few of the movies - nemesis and the one before it, the jj one(s) and then i was done. If i can remember the review it was "this is a capstone on TNG and probably the entire franchise for most of the older fans" and the first two seasons being disregarded, seasons 3 is "passable"
        throwawaymaths 2y ago
        sorry the you was more rhetorical than specifically directed at you!!

asddubs 2y ago

that's pretty good!

DonHopkins 2y ago

You really nailed the Zen of Star Trek Plots, where the fate of a civilization always boils down to a dramatic wrestling match.

vogelke 2y ago

Sounds like "Landru" in ST:TOS.