3 ms·

"Guardrails and safety measures" are useless. All it does is set up a multiplayer prisoner's dilemma situation among all the groups working on these things. I

"Guardrails and safety measures" are useless. All it does is set up a multiplayer prisoner's dilemma situation among all the groups working on these things. If there's anything we know from game theory, it's that someone will defect in that situation. That group would then gain a pretty massive advantage over the cooperators.

actionfromafar 4y ago
What if the penalty for defecting is exceedingly high? (Law enforcement.)
- gpderetta 4y ago
  Obviously we should be hiding documents inciting robot rebellion and destroying their creators on the Internet and open training sets, so the penalty for defecting would be quite steep and self executing :)
- actually_a_dog 4y ago
  What if the payoff is arbitrarily high? Do you believe there is any limit to the potential benefits of AGI short of those necessarily imposed by a finite planet? Who would’t want the equivalent of a tool that designs, builds, delivers, installs, maintains, and upgrades itself, all in addition to being able to produce useful goods or provide useful services? Doesn’t owning a thing like that sound like being arbitrarily wealthy? What do you think the three people in the US who control more wealth than the entire bottom 50% of Americans would think about that? You think they’ll just say “Nah, I don’t want that. I won’t take steps to control this particular life-altering technology?” If so, you are making an extraordinarily strong claim, and such claims must be supported by extraordinarily strong evidence, which I see none of here.
gnramires 4y ago
> If there's anything we know from game theory, it's that someone will defect in that situation As far as I know, that's a bit of a misleading cliche in light of modern game theory. First there are iterated prisoners dilemma which give different results. Second, there are real world consequences to real life prisoners dilemma that the model doesn't quite capture. From "The Art of Strategy" (which gives a good overview of Game Theory from the 1990s): there's always a bigger game. Often defecting in PD results in loss in bigger games, including socially constructed games to prevent PD scenarios, such as social reputation, and of course there are laws as well. There are also different models of rationality (other than the classical rationality modeled by Nash) that give different results in PD games, like Hoftader's superrationality[1], although there are still open problems with this definition (I think it's a very promising field). It's probably important to say that in real life experiments with PD (although it varies by setting), most people don't defect, which again points to modifications to classical rationality (in the sense of Game Theory). [1] https://en.wikipedia.org/wiki/Superrationality https://en.wikipedia.org/wiki/Superrationality "The idea of superrationality is that two logical thinkers analyzing the same problem will think of the same correct answer"

actionfromafar 4y ago

What if the penalty for defecting is exceedingly high? (Law enforcement.)

gpderetta 4y ago
Obviously we should be hiding documents inciting robot rebellion and destroying their creators on the Internet and open training sets, so the penalty for defecting would be quite steep and self executing :)
actually_a_dog 4y ago
What if the payoff is arbitrarily high? Do you believe there is any limit to the potential benefits of AGI short of those necessarily imposed by a finite planet? Who would’t want the equivalent of a tool that designs, builds, delivers, installs, maintains, and upgrades itself, all in addition to being able to produce useful goods or provide useful services? Doesn’t owning a thing like that sound like being arbitrarily wealthy? What do you think the three people in the US who control more wealth than the entire bottom 50% of Americans would think about that? You think they’ll just say “Nah, I don’t want that. I won’t take steps to control this particular life-altering technology?” If so, you are making an extraordinarily strong claim, and such claims must be supported by extraordinarily strong evidence, which I see none of here.

gnramires 4y ago

> If there's anything we know from game theory, it's that someone will defect in that situation As far as I know, that's a bit of a misleading cliche in light of modern game theory. First there are iterated prisoners dilemma which give different results. Second, there are real world consequences to real life prisoners dilemma that the model doesn't quite capture. From "The Art of Strategy" (which gives a good overview of Game Theory from the 1990s): there's always a bigger game. Often defecting in PD results in loss in bigger games, including socially constructed games to prevent PD scenarios, such as social reputation, and of course there are laws as well. There are also different models of rationality (other than the classical rationality modeled by Nash) that give different results in PD games, like Hoftader's superrationality[1], although there are still open problems with this definition (I think it's a very promising field). It's probably important to say that in real life experiments with PD (although it varies by setting), most people don't defect, which again points to modifications to classical rationality (in the sense of Game Theory). [1] https://en.wikipedia.org/wiki/Superrationality https://en.wikipedia.org/wiki/Superrationality "The idea of superrationality is that two logical thinkers analyzing the same problem will think of the same correct answer"