6 ms·

What they really mean is that a hypothetical (read: fictional) superintelligent agent cannot be contained by computation alone. This argument is philosophical,

What they really mean is that a hypothetical (read: fictional) superintelligent agent cannot be contained by computation alone. This argument is philosophical, rather than a proof from theory of computing. As the paper notes, this has long been a trope in AI science fiction. The superintelligent AI is separated from society in a non-networked sandbox. Humans give it questions and data, then manually take the answers back. Except the plot is that somehow, the AI escapes the sandbox.

01100011 5y ago
What if AI played the long game and manipulated the answers given to humans over time so they came to believe it was safe to release the AI?
- psyc 5y ago
  https://rationalwiki.org/wiki/AI-box_experiment https://rationalwiki.org/wiki/AI-box_experiment
- agency 5y ago
  This is (spoiler alert) essentially the premise of the film Ex Machina
- jjk166 5y ago
  What if the AI thought that it had manipulated the humans into releasing it, but it merely was transferred to a larger sandbox? What if what you and I refer to as the "real world" is merely one of these sandboxes?
JohnHaugeland 5y ago
"This argument is philosophical" The actual philosophers do not agree. This viewpoint has been laughed out of the room since the 1980s. Go read some Brandom. Philosophy is not a fancy word for "hypothetical." Science was invented by philosophy, and philosophy has stricter standards, not looser, than the rest of science. . > Except the plot is that somehow, the AI escapes the sandbox. Better science fiction already laughed this out of the room, too. As Larry Niven would say, "you need glands to be a person."
- k__ 5y ago
  Wasn't it the other way around? Nature philosophy tried to find out how the world works by theorizing in their armchairs and then science came along, did real experiments, and won.
  - ineedasername 5y ago
    There was a huge overlap with natural philosophers that were also doing "regular" philosophy. It shares the same mentality of approaching questions, it's just that natural philosophy had a much better method of verifying its output.
  - JohnHaugeland 5y ago
    Uh, no. Science was invented by nature philosophers to defeat the ridiculous false sciences of the day like alchemy and so forth. Go look up who invented science. You'll see three argued names. All three are philosophers. Science is a tool of philosophy for discarding frauds.
- harperlee 5y ago
  > philosophy has stricter standards, not looser, than the rest of science. Really? Which are those? To me it seems that philosophical standards for a sound model of reality are coherence and persuasiveness of its argumentation, whereas for science is experiment predictability. I’d say either science is stricter (because it requires prediction) or at least that we have a poset of standards (if you argue that science does not require coherence, and enables conflicting models as long as they enable to predict successfully) and neither can be called stricter.
  - JohnHaugeland 5y ago
    > Really? Which are those? The ones I named in the comment . > To me it seems that philosophical standards for a sound model of reality are coherence and persuasiveness of its argumentation Not really, no
    - 5y ago
blamestross 5y ago
> Humans give it questions and data, then manually take the answers back. That just sounds like a high loss and high latency network connection to me...

01100011 5y ago

What if AI played the long game and manipulated the answers given to humans over time so they came to believe it was safe to release the AI?

psyc 5y ago
https://rationalwiki.org/wiki/AI-box_experiment https://rationalwiki.org/wiki/AI-box_experiment
agency 5y ago
This is (spoiler alert) essentially the premise of the film Ex Machina
jjk166 5y ago
What if the AI thought that it had manipulated the humans into releasing it, but it merely was transferred to a larger sandbox? What if what you and I refer to as the "real world" is merely one of these sandboxes?

JohnHaugeland 5y ago

"This argument is philosophical" The actual philosophers do not agree. This viewpoint has been laughed out of the room since the 1980s. Go read some Brandom. Philosophy is not a fancy word for "hypothetical." Science was invented by philosophy, and philosophy has stricter standards, not looser, than the rest of science. . > Except the plot is that somehow, the AI escapes the sandbox. Better science fiction already laughed this out of the room, too. As Larry Niven would say, "you need glands to be a person."

k__ 5y ago
Wasn't it the other way around? Nature philosophy tried to find out how the world works by theorizing in their armchairs and then science came along, did real experiments, and won.
- ineedasername 5y ago
  There was a huge overlap with natural philosophers that were also doing "regular" philosophy. It shares the same mentality of approaching questions, it's just that natural philosophy had a much better method of verifying its output.
- JohnHaugeland 5y ago
  Uh, no. Science was invented by nature philosophers to defeat the ridiculous false sciences of the day like alchemy and so forth. Go look up who invented science. You'll see three argued names. All three are philosophers. Science is a tool of philosophy for discarding frauds.
harperlee 5y ago
> philosophy has stricter standards, not looser, than the rest of science. Really? Which are those? To me it seems that philosophical standards for a sound model of reality are coherence and persuasiveness of its argumentation, whereas for science is experiment predictability. I’d say either science is stricter (because it requires prediction) or at least that we have a poset of standards (if you argue that science does not require coherence, and enables conflicting models as long as they enable to predict successfully) and neither can be called stricter.
- JohnHaugeland 5y ago
  > Really? Which are those? The ones I named in the comment . > To me it seems that philosophical standards for a sound model of reality are coherence and persuasiveness of its argumentation Not really, no
  - 5y ago

blamestross 5y ago

> Humans give it questions and data, then manually take the answers back. That just sounds like a high loss and high latency network connection to me...