4 ms·

People really really don’t understand the implications of AGI. Whether or not you believe we will reach it in a fee years, we are certainly wayy closer today t

People really really don’t understand the implications of AGI. Whether or not you believe we will reach it in a fee years, we are certainly wayy closer today than we were even two years ago. The possibility of genuine AGI obliterates all the financial or energy related worries, they pale in comparison to the ultimate impact of such a technology. However, yes, if you believe AGI is not possible or won’t arrive in the coming decade then all the data center buildup seems foolish.

surgical_fire 7mo ago
"any day now" Gotta love this argument. Top it off by saying anyone skeptical is a fool, because of course.
- atleastoptimal 7mo ago
  i’m sure my argument would have no merit if you ignored the thousands of advancements made in AI over the past few years
  - CamperBob2 7mo ago
    I think the biggest case for a bearish attitude towards AGI is simply that we don't take advantage of the intelligence we already have. Look at our elected human leaders, for Pete's sake. If we had access to AGI today, we'd just find novel and interesting ways to ignore it, enslave it, gimp it, and/or bias it.
    - sph 7mo ago
      What, your preference would be to just unleash it upon the world? I wish the average software engineer had any foundation whatsoever in humanities and philosophy before being allowed to make such decisions, but alas, we are doomed.
      - tempodox 7mo ago
        Look at how the feudal tech lords employ the tech we actually have, do you seriously think they suddenly grow a conscience if the mythical AGI came to pass? All observable behavior will be based on insatiable greed for money and power, just as it is now.
  - surgical_fire 7mo ago
    Really? The last major advancement was probably GPT3, af least if we are talking about the LLM companies, the ones involved in the current data center boom. After that was we experienced were marginal improvements of the same technology. Yes, the current models are better than what OpenAI put out at the time of ChatGPT 3, but none of it was revolutionary (and the gains have been less and less perceptible in newer versions). We might be as far from AGI as we were in 2022. I think we are multiple revolutions in technology away from it.
    - CamperBob2 7mo ago
      I think agentic models will prove to be a seismic revolution by themselves, AGI or no AGI. The models themselves may not need to be that much smarter than what we have now. You can make up for a lack of IQ to some extent if you wrap a control loop around a relentless machine.
      - surgical_fire 7mo ago
        That's the thing, we got into hypotheticals. Maybe you are right, maybe not. There were people that dismissed the Internet as a fad. There were people that dismissed Cryptocurrency as a fad. There were people that dismissed NFTs as a fad. I think LLMs are very useful in some contexts, somewhat useless in others, and I think agentic models are highly overrated. Too unreliable to be left by themselves. We are a few revolutions away from that changing, I think. But that, too his hypothetical. What I didn't do is call you a fool for your hypothesis.
tipiirai 7mo ago
I believe AGI is mostly a marketing term, so the data center buildup definitely seems foolish.
danaris 7mo ago
First, you have to define "AGI". Next, you have to have a clear path to reaching it. Then, you have to have the resources to actually walk that path. Only with all three of those can you make any credible claim that AGI is near. As it stands, we have none of them—and the lack of the second is the most damning. It's very, very clear at this point that just scaling up the existing LLMs is not going to reach some critical mass and result in AGI, like the serendipitous sapience of Mycroft in The Moon Is A Harsh Mistress. Given that, any path to AGI necessarily includes some new breakthrough on it (or more than one). And by their essential nature, breakthroughs are not something you can predict or schedule. Indeed, you cannot even be guaranteed that they will ever happen. (It is likely, assuming that it is physically possible to build AGI, that we will figure out how at some point...but not guaranteed.)
- tim333 7mo ago
  On the other hand approx human brain equivalence in computers has been predicted fairly accurately since about 1990 based on Moore's law type projections - mid 2020s by Moravec, 2029 by Kurzweil. The underlying assumption is once the hardware is widely available, hackers will hack something together.
  - danaris 7mo ago
    But Moore's Law went out the window over a decade ago. Oh, sure, we're still getting things smaller and faster—but nowhere near the same rate we were before. Nowadays most (not all, but most) of the advances we're getting are in more and better parallelisation, rather than faster performance on each core. And my (rough, limited) understanding is that based on much more recent projections, it would take several orders of magnitude more computing power to genuinely simulate a human brain than what we have.
    - tim333 7mo ago
      The main Moore's law like metric they track is compute in instructions per second, per dollar. That's kept on a fairly steady exponential for a century or so and doesn't seem to be slowing. (https://www.bvp.com/assets/uploads/2024/03/price-performance-slide-ray-kurzweil-atlas-image.jpg https://www.bvp.com/assets/uploads/2024/03/price-performance...) We are probably not near simulating a brain but near enough for similar functionality. Moravec estimated it by looking at the retina which was fairly well understood and seeing how much compute was needed for similar object detection etc. in his robots.
Der_Einzige 7mo ago
I don't even need AGI/RSI to have unlimited demand for tokens. Spending even a few days with Claude Code/Codex (using opus 4.6 or GPT 5.4) should convince anyone that the demand for tokens will be vast and infinite.

surgical_fire 7mo ago

"any day now" Gotta love this argument. Top it off by saying anyone skeptical is a fool, because of course.

atleastoptimal 7mo ago
i’m sure my argument would have no merit if you ignored the thousands of advancements made in AI over the past few years
- CamperBob2 7mo ago
  I think the biggest case for a bearish attitude towards AGI is simply that we don't take advantage of the intelligence we already have. Look at our elected human leaders, for Pete's sake. If we had access to AGI today, we'd just find novel and interesting ways to ignore it, enslave it, gimp it, and/or bias it.
  - sph 7mo ago
    What, your preference would be to just unleash it upon the world? I wish the average software engineer had any foundation whatsoever in humanities and philosophy before being allowed to make such decisions, but alas, we are doomed.
    - tempodox 7mo ago
      Look at how the feudal tech lords employ the tech we actually have, do you seriously think they suddenly grow a conscience if the mythical AGI came to pass? All observable behavior will be based on insatiable greed for money and power, just as it is now.
- surgical_fire 7mo ago
  Really? The last major advancement was probably GPT3, af least if we are talking about the LLM companies, the ones involved in the current data center boom. After that was we experienced were marginal improvements of the same technology. Yes, the current models are better than what OpenAI put out at the time of ChatGPT 3, but none of it was revolutionary (and the gains have been less and less perceptible in newer versions). We might be as far from AGI as we were in 2022. I think we are multiple revolutions in technology away from it.
  - CamperBob2 7mo ago
    I think agentic models will prove to be a seismic revolution by themselves, AGI or no AGI. The models themselves may not need to be that much smarter than what we have now. You can make up for a lack of IQ to some extent if you wrap a control loop around a relentless machine.
    - surgical_fire 7mo ago
      That's the thing, we got into hypotheticals. Maybe you are right, maybe not. There were people that dismissed the Internet as a fad. There were people that dismissed Cryptocurrency as a fad. There were people that dismissed NFTs as a fad. I think LLMs are very useful in some contexts, somewhat useless in others, and I think agentic models are highly overrated. Too unreliable to be left by themselves. We are a few revolutions away from that changing, I think. But that, too his hypothetical. What I didn't do is call you a fool for your hypothesis.

tipiirai 7mo ago

I believe AGI is mostly a marketing term, so the data center buildup definitely seems foolish.

danaris 7mo ago

First, you have to define "AGI". Next, you have to have a clear path to reaching it. Then, you have to have the resources to actually walk that path. Only with all three of those can you make any credible claim that AGI is near. As it stands, we have none of them—and the lack of the second is the most damning. It's very, very clear at this point that just scaling up the existing LLMs is not going to reach some critical mass and result in AGI, like the serendipitous sapience of Mycroft in The Moon Is A Harsh Mistress. Given that, any path to AGI necessarily includes some new breakthrough on it (or more than one). And by their essential nature, breakthroughs are not something you can predict or schedule. Indeed, you cannot even be guaranteed that they will ever happen. (It is likely, assuming that it is physically possible to build AGI, that we will figure out how at some point...but not guaranteed.)

tim333 7mo ago
On the other hand approx human brain equivalence in computers has been predicted fairly accurately since about 1990 based on Moore's law type projections - mid 2020s by Moravec, 2029 by Kurzweil. The underlying assumption is once the hardware is widely available, hackers will hack something together.
- danaris 7mo ago
  But Moore's Law went out the window over a decade ago. Oh, sure, we're still getting things smaller and faster—but nowhere near the same rate we were before. Nowadays most (not all, but most) of the advances we're getting are in more and better parallelisation, rather than faster performance on each core. And my (rough, limited) understanding is that based on much more recent projections, it would take several orders of magnitude more computing power to genuinely simulate a human brain than what we have.
  - tim333 7mo ago
    The main Moore's law like metric they track is compute in instructions per second, per dollar. That's kept on a fairly steady exponential for a century or so and doesn't seem to be slowing. (https://www.bvp.com/assets/uploads/2024/03/price-performance-slide-ray-kurzweil-atlas-image.jpg https://www.bvp.com/assets/uploads/2024/03/price-performance...) We are probably not near simulating a brain but near enough for similar functionality. Moravec estimated it by looking at the retina which was fairly well understood and seeing how much compute was needed for similar object detection etc. in his robots.

Der_Einzige 7mo ago

I don't even need AGI/RSI to have unlimited demand for tokens. Spending even a few days with Claude Code/Codex (using opus 4.6 or GPT 5.4) should convince anyone that the demand for tokens will be vast and infinite.