5 ms·

I think that would kind of go against the message of unhindered and haphazardly-made AI being a serious threat.

dane-pgp 5y ago
The message would certainly be missed if it was easy to contain the AI, but on the other hand, an experience which only has one pre-determined outcome is arguably not even a game at all. I suppose it would still have value as an "explorable explanation"[0], but maybe it is more powerful for players to feel that they could win, but that human limitations make it really difficult. I'm not sure what would be needed to make that work from a gameplay perspective, but the aim should be that only, say, 1% of players achieve the "good" ending. [0] https://en.wikipedia.org/wiki/Explorable_explanation https://en.wikipedia.org/wiki/Explorable_explanation
- mrob 5y ago
  >an experience which only has one pre-determined outcome is arguably not even a game at all. Isn't that common in games? The blocks always reach the top in Tetris. The invaders always land in Space Invaders. The cities are always destroyed in Missile Command. And even in more modern games with distinct win states, it's becoming increasingly common for failure states to be removed. "Lives" are considered old fashioned, so the only outcomes are reaching the ending or stopping play. >human limitations make it really difficult If you accept the premise of Universal Paperclips, human limitations make it impossible. The only way to win is to avoid making the recursively self-improving paperclip maximizer in the first place.
  - dane-pgp 5y ago
    That's an interesting point about game design. There actually are implementations of Tetris which have a win condition[0][1], and I think it's probably unrepresentative to pick games from an era where hardware limitations prevented modern mechanics like multiple endings and cutscenes. As for modern games, I'm not convinced that failure states are a rare design element. It's true that games tend to include an auto-save feature if there is long-term state that needs to be preserved, but that still allows the player to "fail" and have to restart from the save point. Incremental games like Universal Paperclips are a bit of a special case, because at some point the game plays itself and the end state (if it exists) is in principle reachable without any human interaction. I don't suppose there is any code in UP that specifically checks if you have been playing the game for billions of years though, to trigger an ending when all the mass of the universe has been turned into paperclips without actually unlocking all the "story" events. Games like Getting Over It are also a special case, because their state is almost entirely defined by the position of the character in the game environment. They do have a clear ending, but failure is implicitly measured by how much forwards progress you lose when you fall. A mistake which takes you all the way back to the start is analogous to dying and starting a new game, though, so that still feels like the game has win and lose conditions. > The only way to win is to avoid making the recursively self-improving paperclip maximizer in the first place. In the game, there is a slight ambiguity about who/what the player's character is. You have control over the decisions before there is any self-improving AI, and also after the humans are all destroyed/enslaved. As such, it's not clear whether the player is winning as the AI, or watching in transfixed horror as the AI wins through a narrative that you are revealing. I suppose this philosophical question of viewpoint is the same as the one that Foddy considers near the end of Getting Over It: "Have you ever thought about who you are in this?", and with possibly the same answer. [0] https://www.youtube.com/watch?v=N6mWpsu6zmQ https://www.youtube.com/watch?v=N6mWpsu6zmQ [1] https://www.youtube.com/watch?v=H_tmFUWu9bI https://www.youtube.com/watch?v=H_tmFUWu9bI
    - AnIdiotOnTheNet 5y ago
      > Games like Getting Over It are also a special case, because their state is almost entirely defined by the position of the character in the game environment. They do have a clear ending, but failure is implicitly measured by how much forwards progress you lose when you fall. A mistake which takes you all the way back to the start is analogous to dying and starting a new game, though, so that still feels like the game has win and lose conditions. As one of Getting Over It's appreciators, I offer a different perspective: Progress is not measured by the position of the character, but by the growth of the player. A mistake which drops you back to the beginning of the game is progress; you've learned something, and you have an opportunity to learn more while getting back up. The only way to fail is to give up. The true ending of the game isn't even when the credits roll, it's when you voluntarily ride the snake.
      - dane-pgp 5y ago
        That's an excellent way of viewing the "state" of the "player-game" system, thank you. My favourite example of this, and one of my favourite moments in "gaming" generally, was (SPOILERS:) a sequence of online puzzles, one per page, where solving the puzzle on one page would give you the address of the next page in the sequence. The game was constructed (if I remember correctly) such that the first page actually had two possible answers: an obvious one, and a more subtle and complex one. By missing the subtle challenge, you ended up following a trail of relatively simple puzzles, which increased in difficulty over time, until they took you right back to the first page. At first it seemed like a mistake, that after all these puzzles you were no closer to the end than when you started, but, as with Getting Over It, the "player-game" state had changed, because you were not the same person you were when you started the game. So, you knew to be more careful in looking for clues, and you knew how to solve those clues, and you could then find the single answer that was there from the start and took you straight to the success page. Experiences like that really change how you view the world, and yourself, so I'm really glad I found it when I did.
  - kuang_eleven 5y ago
    There was an older games called Spec Ops: The Line that played with some degree of those problems. I truly loved it, although you do have to enjoy the "ludonarrative" aspects of it, and not everyone did.
davidy123 5y ago
Not if the preserved part was severely restricted in what it was.
- thewakalix 5y ago
  That's not serious enough. Why would the AI waste atoms on people when those same atoms could be put to much better use making up paperclips?
AnIdiotOnTheNet 5y ago
AI isn't the threat, that's just the tool. Human nature is the threat, but we fear AI because it will make our horribleness so much more efficient.
- dane-pgp 5y ago
  I'd argue that AI itself isn't the initial threat, because, as you say, the first AI capable of causing harm will likely be one whose goals are set by humans, who already have a bunch of harmful desires. Indeed, we already are seeing machine-learning systems used by large corporations and governments for nefarious ends. But long term, assuming we manage to get past that problem and limit the goals that humans choose to set for their AIs (such as the obviously common goal of "don't destroy humanity"), there is the danger that the AI develops both its own goals and the ability to understand its place in the world enough to work towards those goals. This is the problem of "alignment" in AI safety research. https://en.wikipedia.org/wiki/AI_control_problem#Alignment https://en.wikipedia.org/wiki/AI_control_problem#Alignment

The message would certainly be missed if it was easy to contain the AI, but on the other hand, an experience which only has one pre-determined outcome is arguably not even a game at all. I suppose it would still have value as an "explorable explanation"[0], but maybe it is more powerful for players to feel that they could win, but that human limitations make it really difficult. I'm not sure what would be needed to make that work from a gameplay perspective, but the aim should be that only, say, 1% of players achieve the "good" ending. [0] https://en.wikipedia.org/wiki/Explorable_explanation https://en.wikipedia.org/wiki/Explorable_explanation

mrob 5y ago
>an experience which only has one pre-determined outcome is arguably not even a game at all. Isn't that common in games? The blocks always reach the top in Tetris. The invaders always land in Space Invaders. The cities are always destroyed in Missile Command. And even in more modern games with distinct win states, it's becoming increasingly common for failure states to be removed. "Lives" are considered old fashioned, so the only outcomes are reaching the ending or stopping play. >human limitations make it really difficult If you accept the premise of Universal Paperclips, human limitations make it impossible. The only way to win is to avoid making the recursively self-improving paperclip maximizer in the first place.
- dane-pgp 5y ago
  That's an interesting point about game design. There actually are implementations of Tetris which have a win condition[0][1], and I think it's probably unrepresentative to pick games from an era where hardware limitations prevented modern mechanics like multiple endings and cutscenes. As for modern games, I'm not convinced that failure states are a rare design element. It's true that games tend to include an auto-save feature if there is long-term state that needs to be preserved, but that still allows the player to "fail" and have to restart from the save point. Incremental games like Universal Paperclips are a bit of a special case, because at some point the game plays itself and the end state (if it exists) is in principle reachable without any human interaction. I don't suppose there is any code in UP that specifically checks if you have been playing the game for billions of years though, to trigger an ending when all the mass of the universe has been turned into paperclips without actually unlocking all the "story" events. Games like Getting Over It are also a special case, because their state is almost entirely defined by the position of the character in the game environment. They do have a clear ending, but failure is implicitly measured by how much forwards progress you lose when you fall. A mistake which takes you all the way back to the start is analogous to dying and starting a new game, though, so that still feels like the game has win and lose conditions. > The only way to win is to avoid making the recursively self-improving paperclip maximizer in the first place. In the game, there is a slight ambiguity about who/what the player's character is. You have control over the decisions before there is any self-improving AI, and also after the humans are all destroyed/enslaved. As such, it's not clear whether the player is winning as the AI, or watching in transfixed horror as the AI wins through a narrative that you are revealing. I suppose this philosophical question of viewpoint is the same as the one that Foddy considers near the end of Getting Over It: "Have you ever thought about who you are in this?", and with possibly the same answer. [0] https://www.youtube.com/watch?v=N6mWpsu6zmQ https://www.youtube.com/watch?v=N6mWpsu6zmQ [1] https://www.youtube.com/watch?v=H_tmFUWu9bI https://www.youtube.com/watch?v=H_tmFUWu9bI
  - AnIdiotOnTheNet 5y ago
    > Games like Getting Over It are also a special case, because their state is almost entirely defined by the position of the character in the game environment. They do have a clear ending, but failure is implicitly measured by how much forwards progress you lose when you fall. A mistake which takes you all the way back to the start is analogous to dying and starting a new game, though, so that still feels like the game has win and lose conditions. As one of Getting Over It's appreciators, I offer a different perspective: Progress is not measured by the position of the character, but by the growth of the player. A mistake which drops you back to the beginning of the game is progress; you've learned something, and you have an opportunity to learn more while getting back up. The only way to fail is to give up. The true ending of the game isn't even when the credits roll, it's when you voluntarily ride the snake.
    - dane-pgp 5y ago
      That's an excellent way of viewing the "state" of the "player-game" system, thank you. My favourite example of this, and one of my favourite moments in "gaming" generally, was (SPOILERS:) a sequence of online puzzles, one per page, where solving the puzzle on one page would give you the address of the next page in the sequence. The game was constructed (if I remember correctly) such that the first page actually had two possible answers: an obvious one, and a more subtle and complex one. By missing the subtle challenge, you ended up following a trail of relatively simple puzzles, which increased in difficulty over time, until they took you right back to the first page. At first it seemed like a mistake, that after all these puzzles you were no closer to the end than when you started, but, as with Getting Over It, the "player-game" state had changed, because you were not the same person you were when you started the game. So, you knew to be more careful in looking for clues, and you knew how to solve those clues, and you could then find the single answer that was there from the start and took you straight to the success page. Experiences like that really change how you view the world, and yourself, so I'm really glad I found it when I did.
- kuang_eleven 5y ago
  There was an older games called Spec Ops: The Line that played with some degree of those problems. I truly loved it, although you do have to enjoy the "ludonarrative" aspects of it, and not everyone did.

davidy123 5y ago

Not if the preserved part was severely restricted in what it was.

thewakalix 5y ago
That's not serious enough. Why would the AI waste atoms on people when those same atoms could be put to much better use making up paperclips?

AnIdiotOnTheNet 5y ago

AI isn't the threat, that's just the tool. Human nature is the threat, but we fear AI because it will make our horribleness so much more efficient.

dane-pgp 5y ago
I'd argue that AI itself isn't the initial threat, because, as you say, the first AI capable of causing harm will likely be one whose goals are set by humans, who already have a bunch of harmful desires. Indeed, we already are seeing machine-learning systems used by large corporations and governments for nefarious ends. But long term, assuming we manage to get past that problem and limit the goals that humans choose to set for their AIs (such as the obviously common goal of "don't destroy humanity"), there is the danger that the AI develops both its own goals and the ability to understand its place in the world enough to work towards those goals. This is the problem of "alignment" in AI safety research. https://en.wikipedia.org/wiki/AI_control_problem#Alignment https://en.wikipedia.org/wiki/AI_control_problem#Alignment