4 ms·

> Lantz enlisted fellow game designer Bennett Foddy to create a simple combat visualizer for late-game battles That's a name I wasn't expecting to see. (But t

> Lantz enlisted fellow game designer Bennett Foddy to create a simple combat visualizer for late-game battles That's a name I wasn't expecting to see. (But that's not you, you're an acrobat. You could swallow a baseball bat.) Anyway, I think what Universal Paperclips is missing is a pacifist mode where you manage to contain the AI and decide to only convert, say, one third of the universe into paperclips, leaving the rest (containing Earth) as some kind of cosmic nature reserve.

skeaker 5y ago
I think that would kind of go against the message of unhindered and haphazardly-made AI being a serious threat.
- dane-pgp 5y ago
  The message would certainly be missed if it was easy to contain the AI, but on the other hand, an experience which only has one pre-determined outcome is arguably not even a game at all. I suppose it would still have value as an "explorable explanation"[0], but maybe it is more powerful for players to feel that they could win, but that human limitations make it really difficult. I'm not sure what would be needed to make that work from a gameplay perspective, but the aim should be that only, say, 1% of players achieve the "good" ending. [0] https://en.wikipedia.org/wiki/Explorable_explanation https://en.wikipedia.org/wiki/Explorable_explanation
  - mrob 5y ago
    >an experience which only has one pre-determined outcome is arguably not even a game at all. Isn't that common in games? The blocks always reach the top in Tetris. The invaders always land in Space Invaders. The cities are always destroyed in Missile Command. And even in more modern games with distinct win states, it's becoming increasingly common for failure states to be removed. "Lives" are considered old fashioned, so the only outcomes are reaching the ending or stopping play. >human limitations make it really difficult If you accept the premise of Universal Paperclips, human limitations make it impossible. The only way to win is to avoid making the recursively self-improving paperclip maximizer in the first place.
    - dane-pgp 5y ago
      That's an interesting point about game design. There actually are implementations of Tetris which have a win condition[0][1], and I think it's probably unrepresentative to pick games from an era where hardware limitations prevented modern mechanics like multiple endings and cutscenes. As for modern games, I'm not convinced that failure states are a rare design element. It's true that games tend to include an auto-save feature if there is long-term state that needs to be preserved, but that still allows the player to "fail" and have to restart from the save point. Incremental games like Universal Paperclips are a bit of a special case, because at some point the game plays itself and the end state (if it exists) is in principle reachable without any human interaction. I don't suppose there is any code in UP that specifically checks if you have been playing the game for billions of years though, to trigger an ending when all the mass of the universe has been turned into paperclips without actually unlocking all the "story" events. Games like Getting Over It are also a special case, because their state is almost entirely defined by the position of the character in the game environment. They do have a clear ending, but failure is implicitly measured by how much forwards progress you lose when you fall. A mistake which takes you all the way back to the start is analogous to dying and starting a new game, though, so that still feels like the game has win and lose conditions. > The only way to win is to avoid making the recursively self-improving paperclip maximizer in the first place. In the game, there is a slight ambiguity about who/what the player's character is. You have control over the decisions before there is any self-improving AI, and also after the humans are all destroyed/enslaved. As such, it's not clear whether the player is winning as the AI, or watching in transfixed horror as the AI wins through a narrative that you are revealing. I suppose this philosophical question of viewpoint is the same as the one that Foddy considers near the end of Getting Over It: "Have you ever thought about who you are in this?", and with possibly the same answer. [0] https://www.youtube.com/watch?v=N6mWpsu6zmQ https://www.youtube.com/watch?v=N6mWpsu6zmQ [1] https://www.youtube.com/watch?v=H_tmFUWu9bI https://www.youtube.com/watch?v=H_tmFUWu9bI
      - AnIdiotOnTheNet 5y ago
        > Games like Getting Over It are also a special case, because their state is almost entirely defined by the position of the character in the game environment. They do have a clear ending, but failure is implicitly measured by how much forwards progress you lose when you fall. A mistake which takes you all the way back to the start is analogous to dying and starting a new game, though, so that still feels like the game has win and lose conditions. As one of Getting Over It's appreciators, I offer a different perspective: Progress is not measured by the position of the character, but by the growth of the player. A mistake which drops you back to the beginning of the game is progress; you've learned something, and you have an opportunity to learn more while getting back up. The only way to fail is to give up. The true ending of the game isn't even when the credits roll, it's when you voluntarily ride the snake.
- davidy123 5y ago
  Not if the preserved part was severely restricted in what it was.
  - thewakalix 5y ago
    That's not serious enough. Why would the AI waste atoms on people when those same atoms could be put to much better use making up paperclips?
- AnIdiotOnTheNet 5y ago
  AI isn't the threat, that's just the tool. Human nature is the threat, but we fear AI because it will make our horribleness so much more efficient.
  - dane-pgp 5y ago
    I'd argue that AI itself isn't the initial threat, because, as you say, the first AI capable of causing harm will likely be one whose goals are set by humans, who already have a bunch of harmful desires. Indeed, we already are seeing machine-learning systems used by large corporations and governments for nefarious ends. But long term, assuming we manage to get past that problem and limit the goals that humans choose to set for their AIs (such as the obviously common goal of "don't destroy humanity"), there is the danger that the AI develops both its own goals and the ability to understand its place in the world enough to work towards those goals. This is the problem of "alignment" in AI safety research. https://en.wikipedia.org/wiki/AI_control_problem#Alignment https://en.wikipedia.org/wiki/AI_control_problem#Alignment
pjc50 5y ago
> only convert, say, one third of the universe into paperclips But then exponential growth stops. A big part of paperclip maximiser discourse is that the maximiser has no other values than expansion. To cease expanding is to die.

skeaker 5y ago

I think that would kind of go against the message of unhindered and haphazardly-made AI being a serious threat.

dane-pgp 5y ago
The message would certainly be missed if it was easy to contain the AI, but on the other hand, an experience which only has one pre-determined outcome is arguably not even a game at all. I suppose it would still have value as an "explorable explanation"[0], but maybe it is more powerful for players to feel that they could win, but that human limitations make it really difficult. I'm not sure what would be needed to make that work from a gameplay perspective, but the aim should be that only, say, 1% of players achieve the "good" ending. [0] https://en.wikipedia.org/wiki/Explorable_explanation https://en.wikipedia.org/wiki/Explorable_explanation
- mrob 5y ago
  >an experience which only has one pre-determined outcome is arguably not even a game at all. Isn't that common in games? The blocks always reach the top in Tetris. The invaders always land in Space Invaders. The cities are always destroyed in Missile Command. And even in more modern games with distinct win states, it's becoming increasingly common for failure states to be removed. "Lives" are considered old fashioned, so the only outcomes are reaching the ending or stopping play. >human limitations make it really difficult If you accept the premise of Universal Paperclips, human limitations make it impossible. The only way to win is to avoid making the recursively self-improving paperclip maximizer in the first place.
  - dane-pgp 5y ago
    That's an interesting point about game design. There actually are implementations of Tetris which have a win condition[0][1], and I think it's probably unrepresentative to pick games from an era where hardware limitations prevented modern mechanics like multiple endings and cutscenes. As for modern games, I'm not convinced that failure states are a rare design element. It's true that games tend to include an auto-save feature if there is long-term state that needs to be preserved, but that still allows the player to "fail" and have to restart from the save point. Incremental games like Universal Paperclips are a bit of a special case, because at some point the game plays itself and the end state (if it exists) is in principle reachable without any human interaction. I don't suppose there is any code in UP that specifically checks if you have been playing the game for billions of years though, to trigger an ending when all the mass of the universe has been turned into paperclips without actually unlocking all the "story" events. Games like Getting Over It are also a special case, because their state is almost entirely defined by the position of the character in the game environment. They do have a clear ending, but failure is implicitly measured by how much forwards progress you lose when you fall. A mistake which takes you all the way back to the start is analogous to dying and starting a new game, though, so that still feels like the game has win and lose conditions. > The only way to win is to avoid making the recursively self-improving paperclip maximizer in the first place. In the game, there is a slight ambiguity about who/what the player's character is. You have control over the decisions before there is any self-improving AI, and also after the humans are all destroyed/enslaved. As such, it's not clear whether the player is winning as the AI, or watching in transfixed horror as the AI wins through a narrative that you are revealing. I suppose this philosophical question of viewpoint is the same as the one that Foddy considers near the end of Getting Over It: "Have you ever thought about who you are in this?", and with possibly the same answer. [0] https://www.youtube.com/watch?v=N6mWpsu6zmQ https://www.youtube.com/watch?v=N6mWpsu6zmQ [1] https://www.youtube.com/watch?v=H_tmFUWu9bI https://www.youtube.com/watch?v=H_tmFUWu9bI
    - AnIdiotOnTheNet 5y ago
      > Games like Getting Over It are also a special case, because their state is almost entirely defined by the position of the character in the game environment. They do have a clear ending, but failure is implicitly measured by how much forwards progress you lose when you fall. A mistake which takes you all the way back to the start is analogous to dying and starting a new game, though, so that still feels like the game has win and lose conditions. As one of Getting Over It's appreciators, I offer a different perspective: Progress is not measured by the position of the character, but by the growth of the player. A mistake which drops you back to the beginning of the game is progress; you've learned something, and you have an opportunity to learn more while getting back up. The only way to fail is to give up. The true ending of the game isn't even when the credits roll, it's when you voluntarily ride the snake.
davidy123 5y ago
Not if the preserved part was severely restricted in what it was.
- thewakalix 5y ago
  That's not serious enough. Why would the AI waste atoms on people when those same atoms could be put to much better use making up paperclips?
AnIdiotOnTheNet 5y ago
AI isn't the threat, that's just the tool. Human nature is the threat, but we fear AI because it will make our horribleness so much more efficient.
- dane-pgp 5y ago
  I'd argue that AI itself isn't the initial threat, because, as you say, the first AI capable of causing harm will likely be one whose goals are set by humans, who already have a bunch of harmful desires. Indeed, we already are seeing machine-learning systems used by large corporations and governments for nefarious ends. But long term, assuming we manage to get past that problem and limit the goals that humans choose to set for their AIs (such as the obviously common goal of "don't destroy humanity"), there is the danger that the AI develops both its own goals and the ability to understand its place in the world enough to work towards those goals. This is the problem of "alignment" in AI safety research. https://en.wikipedia.org/wiki/AI_control_problem#Alignment https://en.wikipedia.org/wiki/AI_control_problem#Alignment

pjc50 5y ago

> only convert, say, one third of the universe into paperclips But then exponential growth stops. A big part of paperclip maximiser discourse is that the maximiser has no other values than expansion. To cease expanding is to die.