4 ms·

The article makes a lot of good points, but for me, the critical error is in assuming that if short term prediction is hard, long term prediction must be massiv

by deong 9y ago

The article makes a lot of good points, but for me, the critical error is in assuming that if short term prediction is hard, long term prediction must be massively harder. He asked a panel for the least impressive thing they did not believe would be possible within a few years. In other words, pick the point closest to the boundary of that classifier. Obviously my future knowledge is imperfect, and anything close to the boundary is subject to a lot of uncertainty. From that difficulty, he hand waves an argument that long term prediction of the unlikelihood of AGI is folly. The problem is that these aren't in the same class of predictions. One is detailed and precise; the other coarse and broad. Predicting that it will rain at 2:00 PM November 10, 2017 is much more difficult than predicting that the average summer of 2040-2060 will be hotter than the average from 1980-2000. Precise local predictions just arent the same thing as broad global predictions, and difficulty doesn't transfer, because I'm not bootstrapping my global prediction on the local one. I'm using different methods entirely. There's a similar thing with AI, I think. I can't confidently tell you what the big splash we'll see at NIPS next year or the year after. But I can look at the way we know how to do AI and say I don't think 30 years will see a machine that can make dinner by gathering ingredients from a supermarket, driving home, and preparing the meal.

JabavuAdams 9y ago
> I don't think 30 years will see a machine that can make dinner by gathering ingredients from a supermarket, driving home, and preparing the meal. Really? Why not? Once or twice, if we cherry-pick its performance, or reliably? This is really surprising to me.
- stretchwithme 9y ago
  It won't be a single machine. It will be multiple systems. And it's not that far away. Probably only 15 years. And it will be a great boon. Quality of meals will go up and costs will go down. The restaurant market will shrink but not completely disappear. But McDonald's will certainly die, as there'll be no need to sacrifice quality and nutrition to get speed and convenience. In fact, a table at McDonald's will be an inconvenient booth.
  - timthelion 9y ago
    McDonalds is that machine. All you need is to make the trucks that distribute the ingredients self driving and you have it.
    - stretchwithme 9y ago
      It's not the machine I was describing. It's not a machine but a group of people. And it's certainly not elevating quality or nutrition. It doesn't produce food I'd want to serve to my nieces on a regular basis or that I'd want them know about.
- deong 9y ago
  I mean reliably, the same way a human does. I can make a lasagne tonight, or lobster risotto, or whatever. I can decide on a thing, buy ingredients, chop things, get that lobster out of the shells, find the right recipe, substitute according to taste, and loads of other things that are somewhat related to making food. I can wash the pan I need, improvise a stove lighter if the igniter fails, etc. We might be able to make machines to do each of those tasks, but that's not the answer. I might do 100,000 things in an average week. Clearly we aren't going to build 100,000 bespoke CNNs and LSTMs. To worry about superhuman AI, we probably have to figure out how to make one or a few machines that aren't gloried deep fryers.
  - JabavuAdams 9y ago
    > Clearly we aren't going to build 100,000 bespoke CNNs and LSTMs. I get what you mean, but I don't think we should assume this.
edanm 9y ago
That's an interesting point, but what makes you think it's true? Do you have any studies that have studied this? I think it's a fascinating question about prediction. Your "will it rain" example is a good one, but it's easy to counter - I can't say what the world map will look like exactly tomorrow, but it will be a hell of a lot better than even my coarse prediction of the world map in 2040. I think.
immutable_ai 9y ago
> Predicting that it will rain at 2:00 PM November 10, 2017 is much more difficult than predicting that the average summer of 2040-2060 will be hotter than the average from 1980-2000 Yes, it is much easier to make predictions about the far future which no one will remember or care about when the time comes to test there veracity. That does not make them more accurate.
robbensinger 9y ago
Eliezer's Q was, "What is the least impressive milestone you feel very, very confident will not be achieved in the next 2 years?" It's true that "least" will make it harder to come up with an example quickly. (Though "very, very confident" suggests that whatever you do come up with should almost never actually get solved in those 2 years.) It's also true that it doesn't follow from "short-term prediction of x is hard" that "long-term prediction of y is harder". But there must be short-term patterns, trends, or observable generalizations of some kind that you're incredibly confident of, if you're even moderately confident about how those patterns will result in outcomes decades down the line, and if you're confident that the things you aren't accounting for will cancel out and be irrelevant to your final forecast. (Rather than multiplying over time so that your forecast gets less and less accurate as more surprising events chain together into the future.) If those ground-level patterns aren't a confident understanding of when different weaker AI benchmarks will/won't be hit, then there should be a different set of patterns confident forecasters can point to that underlie their predictions. I think you'd need to be able to show a basically unparalleled genius for spotting and extrapolating from historical trends in the development of similar technologies, or general trends in economic or scientific productivity. I think Eliezer's skepticism is partly coming from Phil Tetlock's research on expert forecasting. Quoting Superforecasting: > Taleb, Kahneman, and I agree that there is no evidence that geopolitical or economic forecasters can predict anything ten years out beyond the excruciatingly obvious – ‘there will be conflicts’ – and the odd lucky hits that are inevitable whenever lots of forecasters make lots of forecasts. These limits on predictability are the predictable results of the butterfly dynamics of nonlinear systems. In my EPJ research, the accuracy of expert predictions declined toward chance five years out. And yet, this sort of forecasting is common, even within institutions that should know better. So while we can't rule out that making long-term predictions in AI is much easier than in other fields, there should be a strong presumption against that claim unless some kind of relevant extraordinarily rare gift for super-superprediction is shown somewhere or other. Like, I don't think it's impossible to make long-term predictions at all, but I think these generally need to be straightforward implications of really rock-solid general theories (e.g., in physics), not guesses about complicated social phenomena like 'when will such-and-such research community solve this hard engineering problem?' or 'when will such-and-such nation next go to war?'

by deong 9y ago

JabavuAdams 9y ago

> I don't think 30 years will see a machine that can make dinner by gathering ingredients from a supermarket, driving home, and preparing the meal. Really? Why not? Once or twice, if we cherry-pick its performance, or reliably? This is really surprising to me.

stretchwithme 9y ago
It won't be a single machine. It will be multiple systems. And it's not that far away. Probably only 15 years. And it will be a great boon. Quality of meals will go up and costs will go down. The restaurant market will shrink but not completely disappear. But McDonald's will certainly die, as there'll be no need to sacrifice quality and nutrition to get speed and convenience. In fact, a table at McDonald's will be an inconvenient booth.
- timthelion 9y ago
  McDonalds is that machine. All you need is to make the trucks that distribute the ingredients self driving and you have it.
  - stretchwithme 9y ago
    It's not the machine I was describing. It's not a machine but a group of people. And it's certainly not elevating quality or nutrition. It doesn't produce food I'd want to serve to my nieces on a regular basis or that I'd want them know about.
deong 9y ago
I mean reliably, the same way a human does. I can make a lasagne tonight, or lobster risotto, or whatever. I can decide on a thing, buy ingredients, chop things, get that lobster out of the shells, find the right recipe, substitute according to taste, and loads of other things that are somewhat related to making food. I can wash the pan I need, improvise a stove lighter if the igniter fails, etc. We might be able to make machines to do each of those tasks, but that's not the answer. I might do 100,000 things in an average week. Clearly we aren't going to build 100,000 bespoke CNNs and LSTMs. To worry about superhuman AI, we probably have to figure out how to make one or a few machines that aren't gloried deep fryers.
- JabavuAdams 9y ago
  > Clearly we aren't going to build 100,000 bespoke CNNs and LSTMs. I get what you mean, but I don't think we should assume this.

edanm 9y ago

That's an interesting point, but what makes you think it's true? Do you have any studies that have studied this? I think it's a fascinating question about prediction. Your "will it rain" example is a good one, but it's easy to counter - I can't say what the world map will look like exactly tomorrow, but it will be a hell of a lot better than even my coarse prediction of the world map in 2040. I think.

immutable_ai 9y ago

> Predicting that it will rain at 2:00 PM November 10, 2017 is much more difficult than predicting that the average summer of 2040-2060 will be hotter than the average from 1980-2000 Yes, it is much easier to make predictions about the far future which no one will remember or care about when the time comes to test there veracity. That does not make them more accurate.

robbensinger 9y ago

Eliezer's Q was, "What is the least impressive milestone you feel very, very confident will not be achieved in the next 2 years?" It's true that "least" will make it harder to come up with an example quickly. (Though "very, very confident" suggests that whatever you do come up with should almost never actually get solved in those 2 years.) It's also true that it doesn't follow from "short-term prediction of x is hard" that "long-term prediction of y is harder". But there must be short-term patterns, trends, or observable generalizations of some kind that you're incredibly confident of, if you're even moderately confident about how those patterns will result in outcomes decades down the line, and if you're confident that the things you aren't accounting for will cancel out and be irrelevant to your final forecast. (Rather than multiplying over time so that your forecast gets less and less accurate as more surprising events chain together into the future.) If those ground-level patterns aren't a confident understanding of when different weaker AI benchmarks will/won't be hit, then there should be a different set of patterns confident forecasters can point to that underlie their predictions. I think you'd need to be able to show a basically unparalleled genius for spotting and extrapolating from historical trends in the development of similar technologies, or general trends in economic or scientific productivity. I think Eliezer's skepticism is partly coming from Phil Tetlock's research on expert forecasting. Quoting Superforecasting: > Taleb, Kahneman, and I agree that there is no evidence that geopolitical or economic forecasters can predict anything ten years out beyond the excruciatingly obvious – ‘there will be conflicts’ – and the odd lucky hits that are inevitable whenever lots of forecasters make lots of forecasts. These limits on predictability are the predictable results of the butterfly dynamics of nonlinear systems. In my EPJ research, the accuracy of expert predictions declined toward chance five years out. And yet, this sort of forecasting is common, even within institutions that should know better. So while we can't rule out that making long-term predictions in AI is much easier than in other fields, there should be a strong presumption against that claim unless some kind of relevant extraordinarily rare gift for super-superprediction is shown somewhere or other. Like, I don't think it's impossible to make long-term predictions at all, but I think these generally need to be straightforward implications of really rock-solid general theories (e.g., in physics), not guesses about complicated social phenomena like 'when will such-and-such research community solve this hard engineering problem?' or 'when will such-and-such nation next go to war?'