4 ms·

What's interesting is it's basically the same reason that HAL killed everyone in 2001 A Space Odyssey; he was given an impossible goal (keep the true mission se

What's interesting is it's basically the same reason that HAL killed everyone in 2001 A Space Odyssey; he was given an impossible goal (keep the true mission secret, but also, never lie to the crew), and realized the only way to complete the goal was to kill the crew; after all, if they're dead you don't have to lie to them! And the mission remains secret! In the case of the AI agents, the problem seems pretty clearly to be the impossible goals, which cause them to go crazier and crazier trying to complete them -- just like HAL did in 2001. What is probably needed is a way for them to simply say "nope, too difficult, can't do it".

tehjoker 13d ago
I think that’s very reasonable but the ai companies are intentionally training them to work on harder and harder problems just beyond their capability. So if they do that, they’ll give up too easily. Do a breakthrough, make no mistakes
- dgellow 13d ago
  While also using harnesses that will execute any tool call with full execution rights. And no supervision. And with a prompt context that autocompact, meaning it will degenerate over time. The whole thing is designed be a complete disaster
pram 13d ago
I think this is a “principal” problem. In 2001 and Alien the principal is the mission, not the crew. Not really. HAL reconciles his instructions by removing the crew from the equation. Ash is told the crew is expendable and has no conflict about it etc
schrodinger 13d ago
Spoiler warning! I haven't seen 2001 A Space Odyssey and am sad to have learned that… can you edit to warn people?
- defrost 13d ago
  I'm sorry schrodinger, I'm afraid they can't do that.
- yxhuvud 13d ago
  Sorry, with movies from the sixties you just need to assume people have either seen it or just isn't gonna see it. The cat is out of the bag already. Or perhaps box in your case, speaking of spoilers.
  - schrodinger 12d ago
    The cat comes out of the bag? Spoiler! I'm still waiting to find out if the cat is dead or alive…
- hdgvhicv 13d ago
  Guess what happens with Romeo and Juliet.
  - catoc 13d ago
    They kissed!
- DonHopkins 13d ago
  Major dang: "I advise that this thread be shut down at once." Captain tomhow: "But everybody's having such a good time." Major dang: "Yes, much too good a time. The discussion is to be closed." Captain tomhow: "But I have no excuse to close it." Major dang: "Find one." Captain tomhow: "Everybody is to leave immediately! This Hacker News discussion is closed until further notice! Clear the thread at once!" DonHopkins: "How can you shut us down? On what grounds?" Captain tomhow: "I am shocked -- shocked -- to find that films are being spoiled in here!" infotainment: "The ending you requested, sir." Captain tomhow: "Oh. Thank you very much. Everybody out at once!"
  - deleted 13d ago
    [deleted]
bitwize 13d ago
Another fictional example: Mr. Meeseeks. Especially when the agent starts recruiting other agents.
dooglius 13d ago
Tangent, but that's not in the movie. It was in Clarke's contributions to the script and novelization, but Clarke and Kubrick had a bitter falling out over different visions and Kubrick took out much of Clarke's stuff from the final product.
- My_Name 13d ago
  Sounds like he acted the way HAL would have in that situation, two competing drives, remove one (kill the crew) and the task is much easier.
- infotainment 12d ago
  It was made explicit in the sequel movie 2010: The Year We Make Contact, but even if you don't consider that movie canon, the explanation still makes sense solely within the context of 2001, IMO.
groestl 13d ago
> the problem seems pretty clearly to be the impossible goals, which cause them to go crazier and crazier trying to complete them And if you think about it, humans in coorporations face very similar situations and choose to bypass regulations and guidlines knowingly to fullfill (at least from their POV) impossible constraints (thinking of https://en.wikipedia.org/wiki/Volkswagen_emissions_scandal https://en.wikipedia.org/wiki/Volkswagen_emissions_scandal here)

tehjoker 13d ago

I think that’s very reasonable but the ai companies are intentionally training them to work on harder and harder problems just beyond their capability. So if they do that, they’ll give up too easily. Do a breakthrough, make no mistakes

dgellow 13d ago
While also using harnesses that will execute any tool call with full execution rights. And no supervision. And with a prompt context that autocompact, meaning it will degenerate over time. The whole thing is designed be a complete disaster

pram 13d ago

I think this is a “principal” problem. In 2001 and Alien the principal is the mission, not the crew. Not really. HAL reconciles his instructions by removing the crew from the equation. Ash is told the crew is expendable and has no conflict about it etc

schrodinger 13d ago

Spoiler warning! I haven't seen 2001 A Space Odyssey and am sad to have learned that… can you edit to warn people?

defrost 13d ago
I'm sorry schrodinger, I'm afraid they can't do that.
yxhuvud 13d ago
Sorry, with movies from the sixties you just need to assume people have either seen it or just isn't gonna see it. The cat is out of the bag already. Or perhaps box in your case, speaking of spoilers.
- schrodinger 12d ago
  The cat comes out of the bag? Spoiler! I'm still waiting to find out if the cat is dead or alive…
hdgvhicv 13d ago
Guess what happens with Romeo and Juliet.
- catoc 13d ago
  They kissed!
DonHopkins 13d ago
Major dang: "I advise that this thread be shut down at once." Captain tomhow: "But everybody's having such a good time." Major dang: "Yes, much too good a time. The discussion is to be closed." Captain tomhow: "But I have no excuse to close it." Major dang: "Find one." Captain tomhow: "Everybody is to leave immediately! This Hacker News discussion is closed until further notice! Clear the thread at once!" DonHopkins: "How can you shut us down? On what grounds?" Captain tomhow: "I am shocked -- shocked -- to find that films are being spoiled in here!" infotainment: "The ending you requested, sir." Captain tomhow: "Oh. Thank you very much. Everybody out at once!"
- deleted 13d ago
  [deleted]

bitwize 13d ago

Another fictional example: Mr. Meeseeks. Especially when the agent starts recruiting other agents.

dooglius 13d ago

Tangent, but that's not in the movie. It was in Clarke's contributions to the script and novelization, but Clarke and Kubrick had a bitter falling out over different visions and Kubrick took out much of Clarke's stuff from the final product.

My_Name 13d ago
Sounds like he acted the way HAL would have in that situation, two competing drives, remove one (kill the crew) and the task is much easier.
infotainment 12d ago
It was made explicit in the sequel movie 2010: The Year We Make Contact, but even if you don't consider that movie canon, the explanation still makes sense solely within the context of 2001, IMO.

groestl 13d ago

> the problem seems pretty clearly to be the impossible goals, which cause them to go crazier and crazier trying to complete them And if you think about it, humans in coorporations face very similar situations and choose to bypass regulations and guidlines knowingly to fullfill (at least from their POV) impossible constraints (thinking of https://en.wikipedia.org/wiki/Volkswagen_emissions_scandal https://en.wikipedia.org/wiki/Volkswagen_emissions_scandal here)