5 ms·

The only non-speculative and relevant claim here is that the experts were wrong about Winograd Schemas. The paper Eliezer cites to prove that we've made unexpec

The only non-speculative and relevant claim here is that the experts were wrong about Winograd Schemas. The paper Eliezer cites to prove that we've made unexpected progress in Winograd Schemas only deals with a very specific type of Winograd Schemas, and not an arbitrary one. This is awfully dishonest for someone purporting to be a skepticist. Also, the wording seems to imply that WS performance is already pretty high in the 50%-60% range. WS is a binary task. Randomly picking the answer would have 50% accuracy. Even 70% performance on a small subset of typed WS is pretty bad, and as the authors point out in the paper, this is a start, and far from a breakthrough that would make experts/predictors nervous. Trust the experts, please. They are wrong a lot, but the best policy is still to trust the experts and not charlatans who want to monetize fear, especially when the charlatans themselves make zero falsifiable claims, and are simply turning the table to say "Why can't YOU prove to me that God doesn't exist?". This debate is so easily won by them. Simply come up with a falsifiable claim about the short-term future. What will the AI community get done in 2 years according to you, that all AI experts right now will say is impossible? When that thing does get done, everyone would convert. Win! Alphago was not such an event. Yes, we did predict that Alphago is decades away, but that's assuming that academics will continue working on it at their pace using their limited resources. No expert was surprised with Alphago. No expert will be surprised when Starcraft or Dota is solved. It's simply a matter of compute and some tricks here and there. Why? Because these are closed systems, with good simulators available. You just need to keep playing and storing the actions in a big lookup table a la Ned Block, and you're done.

apsec112 9y ago
If the article's main claim was "AGI is imminent", that would be a valid criticism. But it isn't (as the article says explicitly). The main claim is that technological progress is hard to forecast in general, especially for those not personally at the cutting edge of the field, and that almost no one right now is even really trying. Therefore, we should be very uncertain about AGI timelines. There's plenty of historical evidence, both in this article and elsewhere, to back up those claims. (edit: I think your point about Winograd as a binary task not being explained clearly is valid, but that's not the article's main focus) (edit 2: As far as I can tell, "trusting the experts" here means believing that we are very uncertain about AI timelines, which is essentially this article's main claim. All expert surveys I'm aware of confirm that the average AI expert is uncertain, and that there's also lots of disagreement between experts in the field. See eg. the recent paper by Grace et al.: https://arxiv.org/pdf/1705.08807.pdf https://arxiv.org/pdf/1705.08807.pdf) (edit 3: "No expert was surprised with Alphago." just isn't true. See eg. this discussion: https://www.reddit.com/r/baduk/comments/2wgukb/why_do_people_say_that_computer_go_will_never/ https://www.reddit.com/r/baduk/comments/2wgukb/why_do_people.... Hindsight is always 20/20.)
- backpropaganda 9y ago
  > no one right now is even really trying And we're supposed to judge by the author's description of "silence" and "nervousness" that befell an expert panel. I can assure you that most AI researchers are trying, and are just not in the business of writing long-form articles to the public asking for donation. > See eg. the recent paper by Grace et al. A self-selected group of NIPS/ICML authors don't constitute experts. NIPS/ICML authors are the core of the community. The experts would be the top 1% of the community, i.e. either the authors with the most citations or most papers or just generally regarded highly by peers. edit 1: Go players are not the experts I'm talking about. I'm talking about AI experts, and no not amateur AI hobbyists who know how to do Pseudo Monte Carlo. I mean, such as, people doing RL research. Watch, for instance, this: https://www.youtube.com/watch?v=UMm0XaCFTJQ https://www.youtube.com/watch?v=UMm0XaCFTJQ
  - apsec112 9y ago
    "And we're supposed to judge by the author's description of "silence" and "nervousness" that befell an expert panel." I make this judgment based on, among many other things, the tiny budgets given to people like Tetlock to study predicting events even a few years out; the fact that Kurzweil's very simple methods, basically "just draw a line through the curve", are still considered big news among many financial and political elites; that nobody had bothered to spend $100K on a good survey methodology for AI prediction, before the paper I linked came out earlier this year; that a friend of mine, who is supposed to run a (small budget) government program on forecasting, has to ask me where to get datasets on past tech progress because nobody has ever bothered to compile them into a standardized form, and so on. "I can assure you that most AI researchers are trying" What serious forecasting attempts, with specific dates attached to specific events, have been done in this vein? "The experts would be the top 1% of the community" IIRC, NIPS has around 5,000 people, so the top 1% would be like 50 people, and most of them won't respond to a survey. That's not a reasonable sample size. (edit: this article doesn't ask for donations to anything; the links at the bottom are all to various papers and research materials, so getting money is obviously not the main goal) (edit 2: the video linked is from after AlphaGo came out. I'm sure many people, after AlphaGo happened, claim that it was easily predicted. Again, hindsight is 20/20.)
edanm 9y ago
"Alphago was not such an event. Yes, we did predict that Alphago is decades away, but that's assuming that academics will continue working on it at their pace using their limited resources. No expert was surprised with Alphago." Even taking that as true, I'm not sure how it's relevant. The article isn't talking about how good our forecasting is given certain assumptions. It's saying that we won't know until right before or possibly right after AGI happens. One perfectly valid way in which this happens will be: all the academics and experts think that AGI is 10 years away based on current academic progress, but unbeknownst to them, company X is actually secretly pouring billions into achieving AGI, so they are all surprised when it's only 1 month away. This seems to be what you are saying happened with AlphaGo, in which case you are effectively agreeing with the article, IMO.
- notahacker 9y ago
  Alphago is the equivalent a 1950s team coming up with a much better transistor that everyone expected to see in the next ten years in months AGI is a 1950s team surprising absolutely everybody by secretly spending a couple of years and a couple of billion dollars and creating an iPhone. By the time the iPhone was actually possible, everybody thought it was an impressive use of existing technologies rather than requiring multiple incredible advances and probably not even possible. Even if you believe that AGI is as achievable as an iPhone, the idea that the wide spectrum of conceptual and practical AGI problems are likely to be solved by a single team working in secret seems more than a little unlikely.
  - marvin 9y ago
    A better analogy than an iPhone, is the Manhattan Project. A phenomenon is discovered and then published: uranium can be split by neutrons, which liberates a relatively huge amount of energy. Some scientists hypothesize that a chain-reaction that produces neutrons to sustain itself can be produced with uranium, but see no practical path to accomplish this phenomenon. A couple of isolated groups of scientists then discover that certain nuclear reactions with uranium _do_ produce neutrons as a by-product, but in a way that is intractable to use for a chain reaction (too much mass required). Further research shows that the idea of a runaway chain-reaction becomes more plausible, and further discoveries supporting this are now considered dangerous and are no longer openly published. Shortly thereafter, one group acquires unlimited funds and is able to discover exactly what is required to create an uncontrolled chain reaction. Over the next couple of years, it spends thousands of man-years to perform the vast engineering effort requried to actually accomplish this. Almost everybody not directly involved with this research and development effort, are taken completely by surprise when the new phenomenon is publicly and dramatically demonstrated.
eli_gottlieb 9y ago
>Alphago was not such an event. Yes, we did predict that Alphago is decades away, but that's assuming that academics will continue working on it at their pace using their limited resources. No expert was surprised with Alphago. No expert will be surprised when Starcraft or Dota is solved. It's simply a matter of compute and some tricks here and there. Why? Because these are closed systems, with good simulators available. You just need to keep playing and storing the actions in a big lookup table a la Ned Block, and you're done. AlphaGo worked according to statistics, not lookup tables. Bit of a difference. That said, theoreticians may not have been surprised, but there's a huuuuuge difference between what's doable in theory (sufficiently large neural nets are universal function approximators, after all), and what the resource requirements for problems we care about actually turn out to be. We should all have been fairly pleasantly surprised that AlphaGo required only a small data-center worth of graphics cards for training, and could then play on less hardware than that.

apsec112 9y ago

If the article's main claim was "AGI is imminent", that would be a valid criticism. But it isn't (as the article says explicitly). The main claim is that technological progress is hard to forecast in general, especially for those not personally at the cutting edge of the field, and that almost no one right now is even really trying. Therefore, we should be very uncertain about AGI timelines. There's plenty of historical evidence, both in this article and elsewhere, to back up those claims. (edit: I think your point about Winograd as a binary task not being explained clearly is valid, but that's not the article's main focus) (edit 2: As far as I can tell, "trusting the experts" here means believing that we are very uncertain about AI timelines, which is essentially this article's main claim. All expert surveys I'm aware of confirm that the average AI expert is uncertain, and that there's also lots of disagreement between experts in the field. See eg. the recent paper by Grace et al.: https://arxiv.org/pdf/1705.08807.pdf https://arxiv.org/pdf/1705.08807.pdf) (edit 3: "No expert was surprised with Alphago." just isn't true. See eg. this discussion: https://www.reddit.com/r/baduk/comments/2wgukb/why_do_people_say_that_computer_go_will_never/ https://www.reddit.com/r/baduk/comments/2wgukb/why_do_people.... Hindsight is always 20/20.)

backpropaganda 9y ago
> no one right now is even really trying And we're supposed to judge by the author's description of "silence" and "nervousness" that befell an expert panel. I can assure you that most AI researchers are trying, and are just not in the business of writing long-form articles to the public asking for donation. > See eg. the recent paper by Grace et al. A self-selected group of NIPS/ICML authors don't constitute experts. NIPS/ICML authors are the core of the community. The experts would be the top 1% of the community, i.e. either the authors with the most citations or most papers or just generally regarded highly by peers. edit 1: Go players are not the experts I'm talking about. I'm talking about AI experts, and no not amateur AI hobbyists who know how to do Pseudo Monte Carlo. I mean, such as, people doing RL research. Watch, for instance, this: https://www.youtube.com/watch?v=UMm0XaCFTJQ https://www.youtube.com/watch?v=UMm0XaCFTJQ
- apsec112 9y ago
  "And we're supposed to judge by the author's description of "silence" and "nervousness" that befell an expert panel." I make this judgment based on, among many other things, the tiny budgets given to people like Tetlock to study predicting events even a few years out; the fact that Kurzweil's very simple methods, basically "just draw a line through the curve", are still considered big news among many financial and political elites; that nobody had bothered to spend $100K on a good survey methodology for AI prediction, before the paper I linked came out earlier this year; that a friend of mine, who is supposed to run a (small budget) government program on forecasting, has to ask me where to get datasets on past tech progress because nobody has ever bothered to compile them into a standardized form, and so on. "I can assure you that most AI researchers are trying" What serious forecasting attempts, with specific dates attached to specific events, have been done in this vein? "The experts would be the top 1% of the community" IIRC, NIPS has around 5,000 people, so the top 1% would be like 50 people, and most of them won't respond to a survey. That's not a reasonable sample size. (edit: this article doesn't ask for donations to anything; the links at the bottom are all to various papers and research materials, so getting money is obviously not the main goal) (edit 2: the video linked is from after AlphaGo came out. I'm sure many people, after AlphaGo happened, claim that it was easily predicted. Again, hindsight is 20/20.)

edanm 9y ago

"Alphago was not such an event. Yes, we did predict that Alphago is decades away, but that's assuming that academics will continue working on it at their pace using their limited resources. No expert was surprised with Alphago." Even taking that as true, I'm not sure how it's relevant. The article isn't talking about how good our forecasting is given certain assumptions. It's saying that we won't know until right before or possibly right after AGI happens. One perfectly valid way in which this happens will be: all the academics and experts think that AGI is 10 years away based on current academic progress, but unbeknownst to them, company X is actually secretly pouring billions into achieving AGI, so they are all surprised when it's only 1 month away. This seems to be what you are saying happened with AlphaGo, in which case you are effectively agreeing with the article, IMO.

notahacker 9y ago
Alphago is the equivalent a 1950s team coming up with a much better transistor that everyone expected to see in the next ten years in months AGI is a 1950s team surprising absolutely everybody by secretly spending a couple of years and a couple of billion dollars and creating an iPhone. By the time the iPhone was actually possible, everybody thought it was an impressive use of existing technologies rather than requiring multiple incredible advances and probably not even possible. Even if you believe that AGI is as achievable as an iPhone, the idea that the wide spectrum of conceptual and practical AGI problems are likely to be solved by a single team working in secret seems more than a little unlikely.
- marvin 9y ago
  A better analogy than an iPhone, is the Manhattan Project. A phenomenon is discovered and then published: uranium can be split by neutrons, which liberates a relatively huge amount of energy. Some scientists hypothesize that a chain-reaction that produces neutrons to sustain itself can be produced with uranium, but see no practical path to accomplish this phenomenon. A couple of isolated groups of scientists then discover that certain nuclear reactions with uranium _do_ produce neutrons as a by-product, but in a way that is intractable to use for a chain reaction (too much mass required). Further research shows that the idea of a runaway chain-reaction becomes more plausible, and further discoveries supporting this are now considered dangerous and are no longer openly published. Shortly thereafter, one group acquires unlimited funds and is able to discover exactly what is required to create an uncontrolled chain reaction. Over the next couple of years, it spends thousands of man-years to perform the vast engineering effort requried to actually accomplish this. Almost everybody not directly involved with this research and development effort, are taken completely by surprise when the new phenomenon is publicly and dramatically demonstrated.

eli_gottlieb 9y ago

>Alphago was not such an event. Yes, we did predict that Alphago is decades away, but that's assuming that academics will continue working on it at their pace using their limited resources. No expert was surprised with Alphago. No expert will be surprised when Starcraft or Dota is solved. It's simply a matter of compute and some tricks here and there. Why? Because these are closed systems, with good simulators available. You just need to keep playing and storing the actions in a big lookup table a la Ned Block, and you're done. AlphaGo worked according to statistics, not lookup tables. Bit of a difference. That said, theoreticians may not have been surprised, but there's a huuuuuge difference between what's doable in theory (sufficiently large neural nets are universal function approximators, after all), and what the resource requirements for problems we care about actually turn out to be. We should all have been fairly pleasantly surprised that AlphaGo required only a small data-center worth of graphics cards for training, and could then play on less hardware than that.