5 ms·

I haven't read Bostrom's book. I don't think I would enjoy it. Maybe I need to grudgingly read it to be able to respond to what Bostromites say. Here's the thi

by rspeer 10y ago

I haven't read Bostrom's book. I don't think I would enjoy it. Maybe I need to grudgingly read it to be able to respond to what Bostromites say. Here's the thing. If Bostrom's claims about AI are so strong, why does everyone who's referring to his book as their source of beliefs about the future spout non-sequiturs about AI? Here's an example. 80000 Hours has a mission that I generally agree with, to find the most important problems in the world and how people can most effectively work on them. But somehow -- unlike cooler-headed organizations like GiveWell -- they've decided that one of the biggest problems, bigger than malaria, bigger than global warming, is "AI risk" (by which they mean the threat of superhuman AGI, not the real but lesser threat that existing AI could make bad judgments). [1] To illustrate this, they refer to what the wise Professor Bostrom has to say, and then show a video of a current AI playing Space Invaders. "At a super-human level", they say pointedly. What the hell does Space Invaders have to do with artificial general intelligence? For that matter, what the hell does deep learning have to do with AGI? It's the current new algorithmic technique, but why does it tell us any more about AGI than the Fourier Transform or the singular value decomposition? I would say this is a bias toward wanting to believe in AGI, and looking for what exists in the present as evidence of it, despite the lack of any actual connection. Has 80000 Hours been bamboozled into thinking that playing Space Invaders represents intelligence, or are they doing the bamboozling? And if Bostrom is such a great thought leader, why isn't he saying "guys, stop turning my ideas into nonsense"? [1] https://80000hours.org/career-guide/world-problems/#artificial-intelligence-and-the-control-problem https://80000hours.org/career-guide/world-problems/#artifici...

nohat 10y ago
Bostrom is in no way in charge of people who happen to agree with him wrt ai risk. For the book he mostly collected and organized a lot of existing thought on ai risk (not that he hasn't made his own novel contributions). That's very valuable, largely because it makes for a good reference point to contextualize discussion on the topic. Unfortunately the critics don't seem to have read it because (in my experience) they repeat the same objections without reference to the existing responses to those objections. People do sometimes overblow alphago/ dqn playing Atari, but it's not meaningless. These systems (and other deep learning based systems) can truly learn from scratch on a decent variety of environments. One of the most important unknowns is exactly how difficult various cognitive tasks will prove to be for a machine. Each task accomplished is another data point.
- rspeer 10y ago
  I wouldn't say that DeepMind learns Atari games "from scratch" any more than Deep Blue learned chess from scratch. It learns to play Atari games because it's a machine designed to learn to play Atari games.
  - Elrac 10y ago
    I strongly disagree. You don't seem to be aware of the difference in approach between Deep Blue and DeepMind. Deep Blue was hand-led directly and specifically to solve the problem of chess: It was provided with a library of opening moves, some sophisticated tactical algorithms relevant to the problem of chess, a library of strategies for chess, and so on. Many actual human masters of chess were consulted, directly or indirectly, to help with developing Deep Blue's approach to the problem. DeepMind, on the other hand, was created as a "blank slate" with no more hard-wired instruction than "create optimal algorithms to achieve the winning state, given the inputs." Critically, its learning phase is completely self-directed. Essentially, the box is given access to the controls and the video screen content and then sent on its way. It's instructive to note that this is pretty much exactly how, very generally speaking, evolution and intelligence solve the problem of survival: every organism has controls and a glimpse of "game state" and has to learn (collectively as a species, individually as an organism) to play the game successfully.
    - Ngunyan 10y ago
      > DeepMind, on the other hand, was created as a "blank slate" with no more hard-wired instruction than "create optimal algorithms to achieve the winning state, given the inputs." Critically, its learning phase is completely self-directed. Essentially, the box is given access to the controls and the video screen content and then sent on its way. Have you seen DeepMind algorithm to be able to say this ? Are there other people outside of Google who have seen the algorithm and can confirm Google's press release?
      - Elrac 10y ago
        Maybe you have some legitimate concern about Googles' claim as per their press release and my comment. Who knows, maybe they have some reason to lie about what they did! But then I wonder why you aren't asking the same question of my parent poster. Has he viewed the DeepMind code, is he qualified to tell us it works the same as chess code? Having made that claim backed on even less evidence than I made mine, I'd say his burden of proof is somewhat greater.

by rspeer 10y ago

nohat 10y ago

Bostrom is in no way in charge of people who happen to agree with him wrt ai risk. For the book he mostly collected and organized a lot of existing thought on ai risk (not that he hasn't made his own novel contributions). That's very valuable, largely because it makes for a good reference point to contextualize discussion on the topic. Unfortunately the critics don't seem to have read it because (in my experience) they repeat the same objections without reference to the existing responses to those objections. People do sometimes overblow alphago/ dqn playing Atari, but it's not meaningless. These systems (and other deep learning based systems) can truly learn from scratch on a decent variety of environments. One of the most important unknowns is exactly how difficult various cognitive tasks will prove to be for a machine. Each task accomplished is another data point.

rspeer 10y ago
I wouldn't say that DeepMind learns Atari games "from scratch" any more than Deep Blue learned chess from scratch. It learns to play Atari games because it's a machine designed to learn to play Atari games.
- Elrac 10y ago
  I strongly disagree. You don't seem to be aware of the difference in approach between Deep Blue and DeepMind. Deep Blue was hand-led directly and specifically to solve the problem of chess: It was provided with a library of opening moves, some sophisticated tactical algorithms relevant to the problem of chess, a library of strategies for chess, and so on. Many actual human masters of chess were consulted, directly or indirectly, to help with developing Deep Blue's approach to the problem. DeepMind, on the other hand, was created as a "blank slate" with no more hard-wired instruction than "create optimal algorithms to achieve the winning state, given the inputs." Critically, its learning phase is completely self-directed. Essentially, the box is given access to the controls and the video screen content and then sent on its way. It's instructive to note that this is pretty much exactly how, very generally speaking, evolution and intelligence solve the problem of survival: every organism has controls and a glimpse of "game state" and has to learn (collectively as a species, individually as an organism) to play the game successfully.
  - Ngunyan 10y ago
    > DeepMind, on the other hand, was created as a "blank slate" with no more hard-wired instruction than "create optimal algorithms to achieve the winning state, given the inputs." Critically, its learning phase is completely self-directed. Essentially, the box is given access to the controls and the video screen content and then sent on its way. Have you seen DeepMind algorithm to be able to say this ? Are there other people outside of Google who have seen the algorithm and can confirm Google's press release?
    - Elrac 10y ago
      Maybe you have some legitimate concern about Googles' claim as per their press release and my comment. Who knows, maybe they have some reason to lie about what they did! But then I wonder why you aren't asking the same question of my parent poster. Has he viewed the DeepMind code, is he qualified to tell us it works the same as chess code? Having made that claim backed on even less evidence than I made mine, I'd say his burden of proof is somewhat greater.