4 ms·

What if you assume that the AI might be 100x or 1000x smarter than its guards, not just 2x or 3x smarter? And the main reason for developing AIs is to do thing

What if you assume that the AI might be 100x or 1000x smarter than its guards, not just 2x or 3x smarter? And the main reason for developing AIs is to do things in the real world. So, unlike a prisoner in jail, the AI might be advising the government or helping scientific research, which would provide many opportunities for subtle manipulation, even while it is imprisoned

breuleux 12y ago
> What if you assume that the AI might be 100x or 1000x smarter than its guards, not just 2x or 3x smarter? If you can make an AI that's 100x smarter than you are, surely you can make one that's only 2x or 3x smarter. Why not start with the latter? If the experiment turns out well, then you can use the 2x/3x smarter AI to guard a 10x smarter AI, and so on. > And the main reason for developing AIs is to do things in the real world. So, unlike a prisoner in jail, the AI might be advising the government or helping scientific research, which would provide many opportunities for subtle manipulation, even while it is imprisoned Most of the time, it is much harder to come up with solutions to a problem than to verify that the solutions are correct. If you consider NP problems, for instance, we can use the AI to find good solutions heuristically and we can prove the correctness of their solutions with a trivial algorithm. That would still be very useful, but there wouldn't be any room for foul play.
- slayed0 12y ago
  It seems it would be very hard to place an arbitrary restriction like that (2x human intelligence vs 100x human intelligence) on something that is learning. I think the idea here is that the AI would begin to learn on its own and become smarter at a frightening pace, not that we would set out to create something that had exactly 2x human intelligence. How do you quantify human intelligence like that anyway? We are very similar to proposed AI/machine learning techniques in that we form predictive models from observation, but how can that really be measured?
  - breuleux 12y ago
    > I think the idea here is that the AI would begin to learn on its own and become smarter at a frightening pace With what resources? The AI is in a box, remember: we are running it on a machine with only so much memory and processing power. There's only so much it can do within these limitations, it's not like we're giving it a spending account.

breuleux 12y ago

> What if you assume that the AI might be 100x or 1000x smarter than its guards, not just 2x or 3x smarter? If you can make an AI that's 100x smarter than you are, surely you can make one that's only 2x or 3x smarter. Why not start with the latter? If the experiment turns out well, then you can use the 2x/3x smarter AI to guard a 10x smarter AI, and so on. > And the main reason for developing AIs is to do things in the real world. So, unlike a prisoner in jail, the AI might be advising the government or helping scientific research, which would provide many opportunities for subtle manipulation, even while it is imprisoned Most of the time, it is much harder to come up with solutions to a problem than to verify that the solutions are correct. If you consider NP problems, for instance, we can use the AI to find good solutions heuristically and we can prove the correctness of their solutions with a trivial algorithm. That would still be very useful, but there wouldn't be any room for foul play.

slayed0 12y ago
It seems it would be very hard to place an arbitrary restriction like that (2x human intelligence vs 100x human intelligence) on something that is learning. I think the idea here is that the AI would begin to learn on its own and become smarter at a frightening pace, not that we would set out to create something that had exactly 2x human intelligence. How do you quantify human intelligence like that anyway? We are very similar to proposed AI/machine learning techniques in that we form predictive models from observation, but how can that really be measured?
- breuleux 12y ago
  > I think the idea here is that the AI would begin to learn on its own and become smarter at a frightening pace With what resources? The AI is in a box, remember: we are running it on a machine with only so much memory and processing power. There's only so much it can do within these limitations, it's not like we're giving it a spending account.