5 ms·

Something to note about the "we're on an exponential (or at least more-than-linear) increase in technology" arguments: They generally need to reach back quite

Something to note about the "we're on an exponential (or at least more-than-linear) increase in technology" arguments: They generally need to reach back quite a long time to sound convincing. Yudkowsky makes two arguments about how fast technology or society is evolving: in one he chooses 500 years ago, 1517. In another, he talks about "the last 10,000 years." In contrast, Chollet compares 1900-1950 with 1950-2000. I agree, we've changed the world in big ways compared to 1517 or 8000 BC. But if we're on a more-than-linear increase, then we should be seeing more and more technological growth in very recent timeframes, not needing to reach back centuries or millennia. In fact, if you consider a y = log(x) or y = sqrt(x) function, those more closely fit a narrative of "If you look back a long time, things seem almost crazily changed, but if you look into recent history, it looks slower" much better than a y = x^2 or y = e^x function.

mannykannot 9y ago
This should be seen in the context of the article Yudkowsky is replying to. Collet's position is that runaway AGI is impossible, and in support of that, he claims that it is impossible for AI technology to grow super-linearly. In his refutation, Yudkowski has merely to show that it is not beyond the bounds of possibility. Of course, insofar as he is also putting forward the view that the risk is significant, he is also putting forward the opinion that it is quite plausible - at least plausible enough to take the scenario seriously. That argument can survive occasional stalls in the rate of technological advance, if that is indeed what we are seeing.
- goatlover 9y ago
  So the real question is about plausibility, not possibility? It's possible that predatory aliens will show up on our doorstep sometime in the next few decades. But it's not very likely, and not something we need to prepare for.
  - mannykannot 9y ago
    By claiming the impossibility of an AI apocalypse, Chollet attempts to avoid discussing any other possibility / plausibility / likelihood than zero, but claiming impossibility brings with it a burden of proof that his arguments cannot carry. As to whether an AI apocalypse is more or less likely than an invasion by predatory aliens, I would guess that it might be the more likely one, but I wouldn't put much effort into defending that position.
LolWolf 9y ago
Just for funsies (but may answer your question): every (differentiable) function is linear at a small enough time scale. This includes e^x or pretty much every function you listed.
- aetherson 9y ago
  Only on infinitely small subintervals of x.
  - LolWolf 9y ago
    This approximation has vanishing error from a linear function (as the interval decreases) and (most importantly!) we only have finite noisy samples; so it’s essentially indistinguishable (in a statistical way) for a small enough interval given some variance.
- dekhn 9y ago
  sin(x) = x for very small x, this is a useful tool for several proofs.
  - cdancette 9y ago
    for all functions, f(x) = f(0) + f'(0) * x for x very close to 0
    - Smaug123 9y ago
      This is a specific instance of what LolWolf just said in the grandparent.
    - dekhn 9y ago
      yeah I didn't know that part, just the more specific one.
      - jjaredsimpson 9y ago
        It's the first term of the Maclaurin series sin(x) = sin(0) + cos(1)x = 0 + x
benbreen 9y ago
Thank you for pointing this out. As an historian, the appeal to a deep historical argument in these situations drive me nuts. We find ultra-rationalist, data-driven people like Yudkowsky relying on a hugely flawed and imperfect historical record to make confident pronouncements. In my opinion, the historical record from, say, 1517 (or even 1717) is so full of gaps and inconsistencies that it is impossible to make even an order of magnitude level estimate regarding something like rate of technological innovation (or even world GDP, arguably). Economists and social scientists are often guilty of this as well - for instance, I really liked the parts of Pinker's Better Angels of Our Nature that dealt with relatively reliable quantitative data from the 20th and 19th centuries. But once he starts talking about things like the An Lushan rebellion or the fall of the Roman Empire, it's a complete mess from an empirical point of view, doing things like taking the numbers of fatalities cited by contemporary participants in an historical conflict at face value. No self-respecting historian would make such big assumptions from such faulty data. Anyway, I realize this is incidental to the larger argument here but insofar as questions about exponential growth of technology draw on historical arguments, I wanted to throw it out there.
btilly 9y ago
In The Innovator's Dilemma over a dozen technologies were documented that were on exponential improvement curves for a period of decades or more. They generally last until the technology either hits a wall in physical possibility, or the thing being delivered is no longer the primary measure of the technology. Here are some examples off of the top of my head that are happening right now: Operations per second of a CPU (The famous Moore's law) bits stored per dollar of RAM Energy density of batteries Energy produced per dollar of solar panels Here are some examples from the past. Distance a steam ship could travel without refueling Maximum power of a gasoline engine Volume of dirt a hydraulic scoop can pick up The history of technological progress is dominated by exponential curves. Saying that it is logically impossible for the future to be likewise dominated by exponential curves is just silly.
naasking 9y ago
> But if we're on a more-than-linear increase, then we should be seeing more and more technological growth in very recent timeframes, not needing to reach back centuries or millennia. We are. Look at the time frame from the invention of the transistor to everyone doing nearly everything online. It's less than the average human lifetime. Look at the timeline from the first computer program to play chess, to beating chess world champions, to the recent announcement of DeepMind that beat all existing computer programs after teaching itself the game within the span of a few hours. Look at the timeframe of computer programs that take dictation to programs that automatically translate between nearly all commonly used languages on Earth. The same could be said for computer vision, computer music, computers driving, and so on. I think the people skeptical of the intelligence explosion are missing the forest for the trees. Our progress in the last century alone is mind boggling. Certainly we can debate the values of the parameters in the intelligence explosion we're in the midst of, but denying it entirely is silly.
- evgen 9y ago
  > Look at the time frame from the invention of the transistor to everyone doing nearly everything online. It's less than the average human lifetime. Look at the time frame from from the invention of controlled heavier-than-air flight to landing a man on the moon; within a human lifetime and before you were born. Look at the time frame from the point when the vast majority of the human population lived their entire life within a 50 mile radius of where they were born and when fast transportation and global travel exponentially increased the genetic mixing of humanity; within a human lifetime and before our grandparents were born. Look at the time frame from the point when information could travel no faster than the speed of a good horse to the time when information could travel across the ocean in the time it took you to saddle a horse; within a human lifetime and almost two centuries ago. Our progress in the last century is significant, but you over-estimate its importance because you are surrounded by it and have little understanding of the history of technology. Things that may seem trivial or even primitive to you were far more important and world-changing inventions, while a lot of what we currently consider significant advances are only important because, for example, we lived in a time when people played finite games better than machines and were around to see that era end.

mannykannot 9y ago

This should be seen in the context of the article Yudkowsky is replying to. Collet's position is that runaway AGI is impossible, and in support of that, he claims that it is impossible for AI technology to grow super-linearly. In his refutation, Yudkowski has merely to show that it is not beyond the bounds of possibility. Of course, insofar as he is also putting forward the view that the risk is significant, he is also putting forward the opinion that it is quite plausible - at least plausible enough to take the scenario seriously. That argument can survive occasional stalls in the rate of technological advance, if that is indeed what we are seeing.

goatlover 9y ago
So the real question is about plausibility, not possibility? It's possible that predatory aliens will show up on our doorstep sometime in the next few decades. But it's not very likely, and not something we need to prepare for.
- mannykannot 9y ago
  By claiming the impossibility of an AI apocalypse, Chollet attempts to avoid discussing any other possibility / plausibility / likelihood than zero, but claiming impossibility brings with it a burden of proof that his arguments cannot carry. As to whether an AI apocalypse is more or less likely than an invasion by predatory aliens, I would guess that it might be the more likely one, but I wouldn't put much effort into defending that position.

LolWolf 9y ago

Just for funsies (but may answer your question): every (differentiable) function is linear at a small enough time scale. This includes e^x or pretty much every function you listed.

aetherson 9y ago
Only on infinitely small subintervals of x.
- LolWolf 9y ago
  This approximation has vanishing error from a linear function (as the interval decreases) and (most importantly!) we only have finite noisy samples; so it’s essentially indistinguishable (in a statistical way) for a small enough interval given some variance.
dekhn 9y ago
sin(x) = x for very small x, this is a useful tool for several proofs.
- cdancette 9y ago
  for all functions, f(x) = f(0) + f'(0) * x for x very close to 0
  - Smaug123 9y ago
    This is a specific instance of what LolWolf just said in the grandparent.
  - dekhn 9y ago
    yeah I didn't know that part, just the more specific one.
    - jjaredsimpson 9y ago
      It's the first term of the Maclaurin series sin(x) = sin(0) + cos(1)x = 0 + x

benbreen 9y ago

Thank you for pointing this out. As an historian, the appeal to a deep historical argument in these situations drive me nuts. We find ultra-rationalist, data-driven people like Yudkowsky relying on a hugely flawed and imperfect historical record to make confident pronouncements. In my opinion, the historical record from, say, 1517 (or even 1717) is so full of gaps and inconsistencies that it is impossible to make even an order of magnitude level estimate regarding something like rate of technological innovation (or even world GDP, arguably). Economists and social scientists are often guilty of this as well - for instance, I really liked the parts of Pinker's Better Angels of Our Nature that dealt with relatively reliable quantitative data from the 20th and 19th centuries. But once he starts talking about things like the An Lushan rebellion or the fall of the Roman Empire, it's a complete mess from an empirical point of view, doing things like taking the numbers of fatalities cited by contemporary participants in an historical conflict at face value. No self-respecting historian would make such big assumptions from such faulty data. Anyway, I realize this is incidental to the larger argument here but insofar as questions about exponential growth of technology draw on historical arguments, I wanted to throw it out there.

btilly 9y ago

In The Innovator's Dilemma over a dozen technologies were documented that were on exponential improvement curves for a period of decades or more. They generally last until the technology either hits a wall in physical possibility, or the thing being delivered is no longer the primary measure of the technology. Here are some examples off of the top of my head that are happening right now: Operations per second of a CPU (The famous Moore's law) bits stored per dollar of RAM Energy density of batteries Energy produced per dollar of solar panels Here are some examples from the past. Distance a steam ship could travel without refueling Maximum power of a gasoline engine Volume of dirt a hydraulic scoop can pick up The history of technological progress is dominated by exponential curves. Saying that it is logically impossible for the future to be likewise dominated by exponential curves is just silly.

naasking 9y ago

> But if we're on a more-than-linear increase, then we should be seeing more and more technological growth in very recent timeframes, not needing to reach back centuries or millennia. We are. Look at the time frame from the invention of the transistor to everyone doing nearly everything online. It's less than the average human lifetime. Look at the timeline from the first computer program to play chess, to beating chess world champions, to the recent announcement of DeepMind that beat all existing computer programs after teaching itself the game within the span of a few hours. Look at the timeframe of computer programs that take dictation to programs that automatically translate between nearly all commonly used languages on Earth. The same could be said for computer vision, computer music, computers driving, and so on. I think the people skeptical of the intelligence explosion are missing the forest for the trees. Our progress in the last century alone is mind boggling. Certainly we can debate the values of the parameters in the intelligence explosion we're in the midst of, but denying it entirely is silly.

evgen 9y ago
> Look at the time frame from the invention of the transistor to everyone doing nearly everything online. It's less than the average human lifetime. Look at the time frame from from the invention of controlled heavier-than-air flight to landing a man on the moon; within a human lifetime and before you were born. Look at the time frame from the point when the vast majority of the human population lived their entire life within a 50 mile radius of where they were born and when fast transportation and global travel exponentially increased the genetic mixing of humanity; within a human lifetime and before our grandparents were born. Look at the time frame from the point when information could travel no faster than the speed of a good horse to the time when information could travel across the ocean in the time it took you to saddle a horse; within a human lifetime and almost two centuries ago. Our progress in the last century is significant, but you over-estimate its importance because you are surrounded by it and have little understanding of the history of technology. Things that may seem trivial or even primitive to you were far more important and world-changing inventions, while a lot of what we currently consider significant advances are only important because, for example, we lived in a time when people played finite games better than machines and were around to see that era end.