5 ms·

Better quality than most such posts, but still seems to be missing the point. The remarkable thing about Bostrom's book is how well it anticipated the objection

by nohat 9y ago

Better quality than most such posts, but still seems to be missing the point. The remarkable thing about Bostrom's book is how well it anticipated the objections and responded to them, yet no one seems to bother refuting his analysis, they just repeat the same objections. I actually agree with a decent bit of what he says on these points, though his application of these observations is kinda baffling. He makes a lot of misguided claims and implications about what proponents believe. I'll sloppily summarize some objections to his points. 1. This doesn't really bother making an argument against superhuman intelligence. Yes, of course intelligence has many components (depending on how you measure it), but that's not an argument against superhuman intelligence. I'm reminded of the joke paper claiming machines can never surpass human largeness, because what does largeness even mean? Why it could mean height or weight, a combination of features, or even something more abstract, so how can you possibly say a machine is larger than a human? 2. Mainly arguing about the definition of 'general' without even trying to consider what the actual usage by Bostrom et al is (this was in the introduction or first chapter if I recall correctly). I agree that the different modes of thought that AI will likely make possible will probably be very useful and powerful, but that's an argument for superhuman ai. 3. Well he makes his first real claim, and it's a strong one: "the only way to get a very human-like thought process is to run the computation on very human-like wet tissue." He doesn't really explore this, or address the interesting technical questions about limits of computational strata, algorithm efficiency, human biological limitation, etc. 4. Few if any think intelligence is likely to be unbounded. Why are these arguments always 'x not infinite, therefore x already at the maximum?' He also seems to be creating counter examples to himself here. 5. Lots of strong, completely unbacked claims about impossibilities here. Some number of these may be true, but I doubt we have already extracted anything near the maximum possible inference about the physical world from the available data, which is basically what his claims boil down to.

rspeer 9y ago
I haven't read Bostrom's book. I don't think I would enjoy it. Maybe I need to grudgingly read it to be able to respond to what Bostromites say. Here's the thing. If Bostrom's claims about AI are so strong, why does everyone who's referring to his book as their source of beliefs about the future spout non-sequiturs about AI? Here's an example. 80000 Hours has a mission that I generally agree with, to find the most important problems in the world and how people can most effectively work on them. But somehow -- unlike cooler-headed organizations like GiveWell -- they've decided that one of the biggest problems, bigger than malaria, bigger than global warming, is "AI risk" (by which they mean the threat of superhuman AGI, not the real but lesser threat that existing AI could make bad judgments). [1] To illustrate this, they refer to what the wise Professor Bostrom has to say, and then show a video of a current AI playing Space Invaders. "At a super-human level", they say pointedly. What the hell does Space Invaders have to do with artificial general intelligence? For that matter, what the hell does deep learning have to do with AGI? It's the current new algorithmic technique, but why does it tell us any more about AGI than the Fourier Transform or the singular value decomposition? I would say this is a bias toward wanting to believe in AGI, and looking for what exists in the present as evidence of it, despite the lack of any actual connection. Has 80000 Hours been bamboozled into thinking that playing Space Invaders represents intelligence, or are they doing the bamboozling? And if Bostrom is such a great thought leader, why isn't he saying "guys, stop turning my ideas into nonsense"? [1] https://80000hours.org/career-guide/world-problems/#artificial-intelligence-and-the-control-problem https://80000hours.org/career-guide/world-problems/#artifici...
- nohat 9y ago
  Bostrom is in no way in charge of people who happen to agree with him wrt ai risk. For the book he mostly collected and organized a lot of existing thought on ai risk (not that he hasn't made his own novel contributions). That's very valuable, largely because it makes for a good reference point to contextualize discussion on the topic. Unfortunately the critics don't seem to have read it because (in my experience) they repeat the same objections without reference to the existing responses to those objections. People do sometimes overblow alphago/ dqn playing Atari, but it's not meaningless. These systems (and other deep learning based systems) can truly learn from scratch on a decent variety of environments. One of the most important unknowns is exactly how difficult various cognitive tasks will prove to be for a machine. Each task accomplished is another data point.
  - rspeer 9y ago
    I wouldn't say that DeepMind learns Atari games "from scratch" any more than Deep Blue learned chess from scratch. It learns to play Atari games because it's a machine designed to learn to play Atari games.
    - Elrac 9y ago
      I strongly disagree. You don't seem to be aware of the difference in approach between Deep Blue and DeepMind. Deep Blue was hand-led directly and specifically to solve the problem of chess: It was provided with a library of opening moves, some sophisticated tactical algorithms relevant to the problem of chess, a library of strategies for chess, and so on. Many actual human masters of chess were consulted, directly or indirectly, to help with developing Deep Blue's approach to the problem. DeepMind, on the other hand, was created as a "blank slate" with no more hard-wired instruction than "create optimal algorithms to achieve the winning state, given the inputs." Critically, its learning phase is completely self-directed. Essentially, the box is given access to the controls and the video screen content and then sent on its way. It's instructive to note that this is pretty much exactly how, very generally speaking, evolution and intelligence solve the problem of survival: every organism has controls and a glimpse of "game state" and has to learn (collectively as a species, individually as an organism) to play the game successfully.
Veedrac 9y ago
> I'm reminded of the joke paper claiming machines can never surpass human largeness, because what does largeness even mean? Link: https://arxiv.org/pdf/1703.10987.pdf https://arxiv.org/pdf/1703.10987.pdf

by nohat 9y ago

rspeer 9y ago

I haven't read Bostrom's book. I don't think I would enjoy it. Maybe I need to grudgingly read it to be able to respond to what Bostromites say. Here's the thing. If Bostrom's claims about AI are so strong, why does everyone who's referring to his book as their source of beliefs about the future spout non-sequiturs about AI? Here's an example. 80000 Hours has a mission that I generally agree with, to find the most important problems in the world and how people can most effectively work on them. But somehow -- unlike cooler-headed organizations like GiveWell -- they've decided that one of the biggest problems, bigger than malaria, bigger than global warming, is "AI risk" (by which they mean the threat of superhuman AGI, not the real but lesser threat that existing AI could make bad judgments). [1] To illustrate this, they refer to what the wise Professor Bostrom has to say, and then show a video of a current AI playing Space Invaders. "At a super-human level", they say pointedly. What the hell does Space Invaders have to do with artificial general intelligence? For that matter, what the hell does deep learning have to do with AGI? It's the current new algorithmic technique, but why does it tell us any more about AGI than the Fourier Transform or the singular value decomposition? I would say this is a bias toward wanting to believe in AGI, and looking for what exists in the present as evidence of it, despite the lack of any actual connection. Has 80000 Hours been bamboozled into thinking that playing Space Invaders represents intelligence, or are they doing the bamboozling? And if Bostrom is such a great thought leader, why isn't he saying "guys, stop turning my ideas into nonsense"? [1] https://80000hours.org/career-guide/world-problems/#artificial-intelligence-and-the-control-problem https://80000hours.org/career-guide/world-problems/#artifici...

nohat 9y ago
Bostrom is in no way in charge of people who happen to agree with him wrt ai risk. For the book he mostly collected and organized a lot of existing thought on ai risk (not that he hasn't made his own novel contributions). That's very valuable, largely because it makes for a good reference point to contextualize discussion on the topic. Unfortunately the critics don't seem to have read it because (in my experience) they repeat the same objections without reference to the existing responses to those objections. People do sometimes overblow alphago/ dqn playing Atari, but it's not meaningless. These systems (and other deep learning based systems) can truly learn from scratch on a decent variety of environments. One of the most important unknowns is exactly how difficult various cognitive tasks will prove to be for a machine. Each task accomplished is another data point.
- rspeer 9y ago
  I wouldn't say that DeepMind learns Atari games "from scratch" any more than Deep Blue learned chess from scratch. It learns to play Atari games because it's a machine designed to learn to play Atari games.
  - Elrac 9y ago
    I strongly disagree. You don't seem to be aware of the difference in approach between Deep Blue and DeepMind. Deep Blue was hand-led directly and specifically to solve the problem of chess: It was provided with a library of opening moves, some sophisticated tactical algorithms relevant to the problem of chess, a library of strategies for chess, and so on. Many actual human masters of chess were consulted, directly or indirectly, to help with developing Deep Blue's approach to the problem. DeepMind, on the other hand, was created as a "blank slate" with no more hard-wired instruction than "create optimal algorithms to achieve the winning state, given the inputs." Critically, its learning phase is completely self-directed. Essentially, the box is given access to the controls and the video screen content and then sent on its way. It's instructive to note that this is pretty much exactly how, very generally speaking, evolution and intelligence solve the problem of survival: every organism has controls and a glimpse of "game state" and has to learn (collectively as a species, individually as an organism) to play the game successfully.

Veedrac 9y ago

> I'm reminded of the joke paper claiming machines can never surpass human largeness, because what does largeness even mean? Link: https://arxiv.org/pdf/1703.10987.pdf https://arxiv.org/pdf/1703.10987.pdf