8 ms·

https://en.wikipedia.org/wiki/Roy_Amara https://en.wikipedia.org/wiki/Roy_Amara > We tend to overestimate the effect of a technology in the short run and under

https://en.wikipedia.org/wiki/Roy_Amara https://en.wikipedia.org/wiki/Roy_Amara > We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run. Amara's Law is a good one, esp in these circles where we have many people working on cutting edge technology. They often give up because it doesn't see adoption, and then move on to something else. Two or three cycles later, when that technology is finally gaining traction, someone takes some off the shelf components and slaps it together, profiting in ways that the original creators would have liked to have seen. Timing is everything, and the technoseti are often _too early_. But they also underestimate how big something will get. I am a huge Python fan, but I never expected back in 2000 that it would be as big as it is today. Never. Cloud Computing, NLP, etc. Google suffers from this immensely.

nunocoracao 3y ago
Amen. I wonder what "underestimating" something like AGI actually means though...
- nerpderp82 3y ago
  By the time AGI arrives, the world will have already been transformed in ways we cannot fathom by intelligence much lower than AGI. Look at the impact that computing has already had, and that is mostly accounting and simulation. Building and factory automation is on the order of a couple if statements. Most programmers can't implement a PID loop. I am not scared by AGI. But to get to AGI you have to pass through a couple points on a curve that occupy the most powerful in terms of agency, less than AGI. An angry toddler that has > 1TW (terawatts) of power.
  - nunocoracao 3y ago
    Agree, the path might be quite bumpy but so could the endgame. Whether AGI wants to help or kill the human race will depend on how aligned it is to our goals. Now, since "our" goals will depend on whoever has the metaphysical ear of such AI this could be problematic.
    - visarga 3y ago
      I think I read this kind of discussions too many times, imagine how the training set of GPT-5 will look like? all sorts of theories of "whether AGI wants to help or kill the human race" spread over the internet and analyzed to death. It is going to know this topic in-and-out. Will be able to write a masterful dissertation on the topic.
      - ben_w 3y ago
        Yes, and that's good because GPT-5 is going to be coming from a team that cares about alignment and which has values only a little different from mine. They can still mess up: https://youtu.be/qV_rOlHjvvs?si=6SI_abC9BLHGoWjt https://youtu.be/qV_rOlHjvvs?si=6SI_abC9BLHGoWjt OTOH, an LLM trained by the FSB or CIA to help their agents, with no limits on what the agents are allowed to get as help because their bosses trust it's always in the service of the nation, will quite "happily" write that essay… and almost certainly get leaked eventually — 10 years, 50 years, doesn't matter — at which point someone who isn't so trustworthy will ask for a doomsday plan and it will be equally "happy" to write that too. Hopefully the agencies like the FSB and CIA aren't quite as bad as I fear, but secrecy and what little has become public makes it impossible to trust them.
    - nerpderp82 3y ago
      AGI doesn't matter, because the world will have been turned upside down multiple times before it even arrives. AI due to its extreme breadth is already smarter than any individual human across all of AIs competencies.
    - Terr_ 3y ago
      I feel that even in just the phrasing of the question, "wants" is doing an awful lot of heavy lifting.
    - Terr_ 3y ago
      I feel that even in just the phrasing of the question, "wants" is doing an awful lot of heavy lifting. It's almost tied in a kind of "No True AGI" framing, where if it doesn't want something it's not really AGI, etc.
  - girvo 3y ago
    > Most programmers can't implement a PID loop. Wait, really? I would expect that most programmers could absolutely implement one if they understood what they are, but perhaps I'm biased from living in embedded land
- ben_w 3y ago
  Assuming it's aligned and we're still around… AGI? Lots of people are still talking like it will be an assistant, perhaps using the "centaur" metaphor, and refusing to believe that it could[0] result in everyone's mental capacities having all the economic relevance of equine muscle power. ASI? We don't know how far above us IQ can go, but it seems reasonable to guess that the smallest possible margin above us is such that even asking that question is like asking the question is akin to asking Chimpanzees to imagine the moon landing, while at worst it's like asking your lawn the same question. [0] I say "could" rather than "will" because there's always the question of "how much does it cost to run that software?" — but the cost of 3430 W at $0.03/kWh matches the World Bank's international poverty line inflation adjusted to 2022: https://www.wolframalpha.com/input?i=%283430+watts+*+%240.03%2FkWh%29+in+per+day https://www.wolframalpha.com/input?i=%283430+watts+*+%240.03...
  - darkerside 3y ago
    Why do we act like people all have an approximately uniform intelligence that will be surpassed by AI intelligence by orders of magnitude in all dimensions? In my experience, different people are smart about different things, sometimes yes by orders of magnitude. Couldn't an AI being smarter than some people be no more surprising than some people being smarter than some other people?
    - ben_w 3y ago
      > Why do we act like people all have an approximately uniform intelligence that will be surpassed by AI intelligence by orders of magnitude in all dimensions? I at least don't consider people to be uniform. If I act like I do, that's illusory. For orders of magnitude higher performance by AI: all have a massive intrinsic speed advantage already (which is currently balanced by an almost but not quite equally massive training set required disadvantage — they can read the entire internet, but they need to too), and the go and chess AI have already vastly exceeded human capacity. I think this kind of thing is also part of why AGI means different things to different people. By my standards, ChatGPT is already an AGI, it's able to operate on text-based tasks in general (the G in AGI) at a level higher than some humans and lower than others — if I'm right, a lot of people are already no longer capable of being economically viable, and their only hope in the current economic system is that the market for their labour doesn't act with perfect information. I may also just be wildly over-estimating the capabilities of ChatGPT.
__loam 3y ago
Python's popularity is more tragedy than success story.
- nerpderp82 3y ago
  I love Python and I agree. Standard ML or OCaml would have been far superior on nearly every axis. Every language is slowly becoming OCaml anyway.
rossdavidh 3y ago
While this is true, there is also a survivorship bias effect. Remember the semantic web? XML? When javascript was going to displace all other scripting languages, from backend to database queries? If a technology follows the trend described by Amara's Law, we will recognize this, because at the end it's bigger than we had imagined (by definition). When it doesn't, we are less likely to remember how those overhyped predictions just never came to pass, because of that very fact. Maybe too early to say if blockchain will turn out to follow Amara's Law, but I have my doubts. Someone using Amara's Law in the first or second AI winters (https://en.wikipedia.org/wiki/AI_winter https://en.wikipedia.org/wiki/AI_winter) would, of course, have the option of just continually saying "it hasn't been long enough yet". But, you know, that very fact means that Amara's Law doesn't tell you much, even in the cases where it turns out the be true. And it doesn't, usually, turn out to be true.

nunocoracao 3y ago

Amen. I wonder what "underestimating" something like AGI actually means though...

nerpderp82 3y ago
By the time AGI arrives, the world will have already been transformed in ways we cannot fathom by intelligence much lower than AGI. Look at the impact that computing has already had, and that is mostly accounting and simulation. Building and factory automation is on the order of a couple if statements. Most programmers can't implement a PID loop. I am not scared by AGI. But to get to AGI you have to pass through a couple points on a curve that occupy the most powerful in terms of agency, less than AGI. An angry toddler that has > 1TW (terawatts) of power.
- nunocoracao 3y ago
  Agree, the path might be quite bumpy but so could the endgame. Whether AGI wants to help or kill the human race will depend on how aligned it is to our goals. Now, since "our" goals will depend on whoever has the metaphysical ear of such AI this could be problematic.
  - visarga 3y ago
    I think I read this kind of discussions too many times, imagine how the training set of GPT-5 will look like? all sorts of theories of "whether AGI wants to help or kill the human race" spread over the internet and analyzed to death. It is going to know this topic in-and-out. Will be able to write a masterful dissertation on the topic.
    - ben_w 3y ago
      Yes, and that's good because GPT-5 is going to be coming from a team that cares about alignment and which has values only a little different from mine. They can still mess up: https://youtu.be/qV_rOlHjvvs?si=6SI_abC9BLHGoWjt https://youtu.be/qV_rOlHjvvs?si=6SI_abC9BLHGoWjt OTOH, an LLM trained by the FSB or CIA to help their agents, with no limits on what the agents are allowed to get as help because their bosses trust it's always in the service of the nation, will quite "happily" write that essay… and almost certainly get leaked eventually — 10 years, 50 years, doesn't matter — at which point someone who isn't so trustworthy will ask for a doomsday plan and it will be equally "happy" to write that too. Hopefully the agencies like the FSB and CIA aren't quite as bad as I fear, but secrecy and what little has become public makes it impossible to trust them.
  - nerpderp82 3y ago
    AGI doesn't matter, because the world will have been turned upside down multiple times before it even arrives. AI due to its extreme breadth is already smarter than any individual human across all of AIs competencies.
  - Terr_ 3y ago
    I feel that even in just the phrasing of the question, "wants" is doing an awful lot of heavy lifting.
  - Terr_ 3y ago
    I feel that even in just the phrasing of the question, "wants" is doing an awful lot of heavy lifting. It's almost tied in a kind of "No True AGI" framing, where if it doesn't want something it's not really AGI, etc.
- girvo 3y ago
  > Most programmers can't implement a PID loop. Wait, really? I would expect that most programmers could absolutely implement one if they understood what they are, but perhaps I'm biased from living in embedded land
ben_w 3y ago
Assuming it's aligned and we're still around… AGI? Lots of people are still talking like it will be an assistant, perhaps using the "centaur" metaphor, and refusing to believe that it could[0] result in everyone's mental capacities having all the economic relevance of equine muscle power. ASI? We don't know how far above us IQ can go, but it seems reasonable to guess that the smallest possible margin above us is such that even asking that question is like asking the question is akin to asking Chimpanzees to imagine the moon landing, while at worst it's like asking your lawn the same question. [0] I say "could" rather than "will" because there's always the question of "how much does it cost to run that software?" — but the cost of 3430 W at $0.03/kWh matches the World Bank's international poverty line inflation adjusted to 2022: https://www.wolframalpha.com/input?i=%283430+watts+*+%240.03%2FkWh%29+in+per+day https://www.wolframalpha.com/input?i=%283430+watts+*+%240.03...
- darkerside 3y ago
  Why do we act like people all have an approximately uniform intelligence that will be surpassed by AI intelligence by orders of magnitude in all dimensions? In my experience, different people are smart about different things, sometimes yes by orders of magnitude. Couldn't an AI being smarter than some people be no more surprising than some people being smarter than some other people?
  - ben_w 3y ago
    > Why do we act like people all have an approximately uniform intelligence that will be surpassed by AI intelligence by orders of magnitude in all dimensions? I at least don't consider people to be uniform. If I act like I do, that's illusory. For orders of magnitude higher performance by AI: all have a massive intrinsic speed advantage already (which is currently balanced by an almost but not quite equally massive training set required disadvantage — they can read the entire internet, but they need to too), and the go and chess AI have already vastly exceeded human capacity. I think this kind of thing is also part of why AGI means different things to different people. By my standards, ChatGPT is already an AGI, it's able to operate on text-based tasks in general (the G in AGI) at a level higher than some humans and lower than others — if I'm right, a lot of people are already no longer capable of being economically viable, and their only hope in the current economic system is that the market for their labour doesn't act with perfect information. I may also just be wildly over-estimating the capabilities of ChatGPT.

__loam 3y ago

Python's popularity is more tragedy than success story.

nerpderp82 3y ago
I love Python and I agree. Standard ML or OCaml would have been far superior on nearly every axis. Every language is slowly becoming OCaml anyway.

rossdavidh 3y ago

While this is true, there is also a survivorship bias effect. Remember the semantic web? XML? When javascript was going to displace all other scripting languages, from backend to database queries? If a technology follows the trend described by Amara's Law, we will recognize this, because at the end it's bigger than we had imagined (by definition). When it doesn't, we are less likely to remember how those overhyped predictions just never came to pass, because of that very fact. Maybe too early to say if blockchain will turn out to follow Amara's Law, but I have my doubts. Someone using Amara's Law in the first or second AI winters (https://en.wikipedia.org/wiki/AI_winter https://en.wikipedia.org/wiki/AI_winter) would, of course, have the option of just continually saying "it hasn't been long enough yet". But, you know, that very fact means that Amara's Law doesn't tell you much, even in the cases where it turns out the be true. And it doesn't, usually, turn out to be true.