5 ms·

His predictions: >AGI by 2027 is strikingly plausible. GPT-2 to GPT-4 took us from ~preschooler to ~smart high-schooler abilities in 4 years. Tracing trendline

by z7 2y ago

His predictions: >AGI by 2027 is strikingly plausible. GPT-2 to GPT-4 took us from ~preschooler to ~smart high-schooler abilities in 4 years. Tracing trendlines in compute (~0.5 orders of magnitude or OOMs/year), algorithmic efficiencies (~0.5 OOMs/year), and “unhobbling” gains (from chatbot to agent), we should expect another preschooler-to-high-schooler-sized qualitative jump by 2027. >I make the following claim: it is strikingly plausible that by 2027, models will be able to do the work of an AI researcher/engineer. That doesn’t require believing in sci-fi; it just requires believing in straight lines on a graph.

zurfer 2y ago
I share the authors view that if things align and compound we are in for a very wild ride. However, I feel the bull case is over emphasized. Not all likely "walls" are sufficiently discussed, some that I think about: - will AI blow past human general intelligence or level off? (because of diminishing returns and "weak" human teachers) - can AI really achieve online learning (economically)? the current architecture can't (just shoving things into large context windows doesn't seem to be enough) - do we as a society tolerate this rapid change or will there be interventions? (protest because people don't want to lose their jobs)
AnimalMuppet 2y ago
> Tracing trendlines in compute (~0.5 orders of magnitude or OOMs/year) 0.5 orders of magnitude is about a factor of 3. We're growing by a factor of 3 in compute every year? I find that rather hard to believe. Oh, echelon points out the quote: > Over the past year, the talk of the town has shifted from $10 billion compute clusters to $100 billion clusters to trillion-dollar clusters. That's how they're getting a factor of 3 in compute every year - by spending more money every year. That's... probably not sustainable for four more years. If it is, it's not sustainable much past that. You're probably not going to see a ten-trillion-dollar cluster; you're definitely not going to see a one-hundred-trillion-dollar cluster, ever. > That doesn’t require believing in sci-fi; it just requires believing in straight lines on a graph. I don't believe in straight lines on a graph. Or rather, I believe that lines are straight over a short enough time span, but may not continue to be straight for, say, twice that time span. Or, they may stay straight for that long. But I don't rate the plausibility as high as he does. (Especially on the "spend more to increase compute" front.)
- zooq_ai 2y ago
  If a $100 Trillion Cluster returns $200 Trillion revenues (say, in less than 7 years), there will be $100 Trillion Clusters
  - AnimalMuppet 2y ago
    $100 trillion is the entire world GDP. I don't care how much debt you're willing to take on or how much stock you sell, you're not going to take the entire world output. What percent of world GDP do you think is realistic to capture for this? As a ballpark estimate, I would say 5% - and even that is only realistic if you're the only one trying to do it. But even at that rate, it's going to take 20 years to build it. Well, if it takes 20 years to build it and then you get a return in 7 years, will the money be there to do it? A 27-year return is a lot different from a 7-year return. So, no, there will not be $100 trillion clusters.
    - zooq_ai 2y ago
      Money is an abstract concept. Net Debt of this planet is Zero. If there is ROI for $100 Trillion, we (banks) will print $100 Trillion by a click of a button Thought Experiment. If by a magic wand the world's productivity doubles overnight. How much extra money needs to be printed to maintain price levels? Ding, Ding, Ding.... $100 Trillion (yes, there is velocity of money and stuff). But, if a project has positive NPV, we can print money (and it's the right thing to do). The worst investors are people who don't get this simple concept and are scared by Debt and Inflation
      - AnimalMuppet 2y ago
        > If there is ROI for $100 Trillion, we (banks) will print $100 Trillion by a click of a button And that will create exactly zero new hardware! All it will do is create twice as much money, so everything will now cost twice as much. As you say, money is an abstract concept. So don't think about money except as a measuring stick for stuff. Is there enough hardware for a cluster that measures at $100 trillion? No, there isn't. There won't be four years from now, or eight years from now. There's not enough hardware. Printing an additional $100 trillion changes the length of the measuring stick, but it doesn't give us any more hardware.

by z7 2y ago

zurfer 2y ago

I share the authors view that if things align and compound we are in for a very wild ride. However, I feel the bull case is over emphasized. Not all likely "walls" are sufficiently discussed, some that I think about: - will AI blow past human general intelligence or level off? (because of diminishing returns and "weak" human teachers) - can AI really achieve online learning (economically)? the current architecture can't (just shoving things into large context windows doesn't seem to be enough) - do we as a society tolerate this rapid change or will there be interventions? (protest because people don't want to lose their jobs)

AnimalMuppet 2y ago

> Tracing trendlines in compute (~0.5 orders of magnitude or OOMs/year) 0.5 orders of magnitude is about a factor of 3. We're growing by a factor of 3 in compute every year? I find that rather hard to believe. Oh, echelon points out the quote: > Over the past year, the talk of the town has shifted from $10 billion compute clusters to $100 billion clusters to trillion-dollar clusters. That's how they're getting a factor of 3 in compute every year - by spending more money every year. That's... probably not sustainable for four more years. If it is, it's not sustainable much past that. You're probably not going to see a ten-trillion-dollar cluster; you're definitely not going to see a one-hundred-trillion-dollar cluster, ever. > That doesn’t require believing in sci-fi; it just requires believing in straight lines on a graph. I don't believe in straight lines on a graph. Or rather, I believe that lines are straight over a short enough time span, but may not continue to be straight for, say, twice that time span. Or, they may stay straight for that long. But I don't rate the plausibility as high as he does. (Especially on the "spend more to increase compute" front.)

zooq_ai 2y ago
If a $100 Trillion Cluster returns $200 Trillion revenues (say, in less than 7 years), there will be $100 Trillion Clusters
- AnimalMuppet 2y ago
  $100 trillion is the entire world GDP. I don't care how much debt you're willing to take on or how much stock you sell, you're not going to take the entire world output. What percent of world GDP do you think is realistic to capture for this? As a ballpark estimate, I would say 5% - and even that is only realistic if you're the only one trying to do it. But even at that rate, it's going to take 20 years to build it. Well, if it takes 20 years to build it and then you get a return in 7 years, will the money be there to do it? A 27-year return is a lot different from a 7-year return. So, no, there will not be $100 trillion clusters.
  - zooq_ai 2y ago
    Money is an abstract concept. Net Debt of this planet is Zero. If there is ROI for $100 Trillion, we (banks) will print $100 Trillion by a click of a button Thought Experiment. If by a magic wand the world's productivity doubles overnight. How much extra money needs to be printed to maintain price levels? Ding, Ding, Ding.... $100 Trillion (yes, there is velocity of money and stuff). But, if a project has positive NPV, we can print money (and it's the right thing to do). The worst investors are people who don't get this simple concept and are scared by Debt and Inflation
    - AnimalMuppet 2y ago
      > If there is ROI for $100 Trillion, we (banks) will print $100 Trillion by a click of a button And that will create exactly zero new hardware! All it will do is create twice as much money, so everything will now cost twice as much. As you say, money is an abstract concept. So don't think about money except as a measuring stick for stuff. Is there enough hardware for a cluster that measures at $100 trillion? No, there isn't. There won't be four years from now, or eight years from now. There's not enough hardware. Printing an additional $100 trillion changes the length of the measuring stick, but it doesn't give us any more hardware.