4 ms·

This is our future: short term growth in economy and stock markets followed by massive layoffs followed by economic crash followed by societal collapse. If you

This is our future: short term growth in economy and stock markets followed by massive layoffs followed by economic crash followed by societal collapse. If you think this is all BS and you will be always smarter than any machine, realize that raw computing power grows by 10^6 every ~25 years. AI seems to follow the same growth. If you think you are safe, read what Geoffrey Hinton, one of the fathers of ML, said about 10-20% chance of AI leading to human extinction in 30 years. Or read about Sam Altman building his bunker in New Zealand while espousing AI utopia. If this doesn't give you shivers, enjoy your blissful ignorance.

ahartmetz 2y ago
>raw computing power grows by 10^6 every ~25 years That is really not true anymore unless there will be a new technology that can do much better than silicon electronics. Doesn't mean that were safe, but it's not that clear. Edit: to expand, a single transistor is made of a pretty pedestrian number of atoms these days. The gate pitch of TSMC "3 nm" is about 40 nm (not all of which is transistor) and silicon atoms are about 0.2 nm apart. So you have, let's say, 30^3 / 0.2^3 ~= 3 million atoms in a transistor. There is no room for another 10^6 improvement because a transistor with just a few atoms probably wouldn't work, the thermal noise would dominate it. Paraphrasing the famous Feynman quote, there isn't much room at the botton anymore, now.
- shalabajser 2y ago
  Your point is true but it is too narrow. A similar argument was bandied about regarding the CPU clock which hit a wall which was defeated with multi-cores and parallelization. The atom wall you are talking about is crushed with 3D semiconductors (hybrid bonding, wafer to wafer..., HBM), wafer scale CPUs like Cerebras, larger clusters, faster networking... Incidentally, massive parallelization suits AI just fine. Do you seriously think that computing power will stop growing? Ever?
  - ahartmetz 2y ago
    I do think so. Every exponential growth is a sigmoid curve on a long enough timescale. The end of Moore's Law is here (for now) even with multicores and everything else, by the way. That you can get more compute by throwing more resources at the problem doesn't really count. The big deal about Moore's Law was that it was free. The 3D stuff, clusters, etc, can increase the number of transistors... but it's gonna cost proportionally more in fabrication and energy when running.
yladiz 2y ago
> raw computing power grows by 10^6 every ~25 years Good thing that the number of neurons in our brains are so many orders of magnitude larger than this that it doesn't matter. I think what you're getting at is that in the next few years we're going to have AGI, which will eventually cause societal collapse. Under that assumption, while all of the AGI arguments are pretty poor, computing power increases over time is one of the weakest. It's going to take so much more than just computing power to get us close to AGI, it's going to take a lot more than LLMs, and it's possible to poison models with specially created data, so I'm not worried; I don't have shivers. What gives me more shivers is that there are people that seriously believe the hype to the degree that they believe the current technology, just a bit improved, will lead to the end of our current life as we know it.
- cheevly 2y ago
  Like 80% of the human brain is used for things that AI does not need (digestion, breathing, etc). So I think that automatically disqualifies your stance, but I am open-minded and would like to know what percent of the human brain you think is vital for AI to outperform us.
  - EA-3167 2y ago
    Even if you discount the role of the whole body as an integrated system, signaling and talking to itself, coordinated by the brain... you're still talking about 16 billion neurons in the cortex alone. That may sound like a number we can approach or exceed with modern tech, but neurons are not transistors. Those 16 billion neurons in the cortex form over 140 TRILLION connections through synapes. Those are both computing elements as well as memory. There's no comparison, not even close. As a bonus we don't each require a giant warehouse to house us, we don't require a vast supply of water, nor do we require a personal dedicated power plant.
    - sixstringtheory 2y ago
      Your last paragraph is the perfect rebuttal to GP. Computers don’t exist in a vacuum. They still require mineral and electrical inputs, repair, orchestration… sounds an awful lot like our bodies!
pavel_lishin 2y ago
> raw computing power grows by 10^6 every ~25 years. For how many years has this been the trend?
b112 2y ago
Raw compute power won't make AGI happen.
aithrowawaycomm 2y ago
It’s not about computing power, it’s the inadequate algorithms, and ultimately the lack of scientific knowledge. Nobody has figured out how animal brains provide the plain old common sense you’d find in a fish or a frog, nor do we have any idea how to emulate that in a machine, even though in pure bits/second a 2025 supercomputer easily outperforms a frog. It’s not just robotics or 3D navigation: Veo2 still screws up object permanence, o1-pro still screws up distinguishing small quantities[1], and 99% of AI agents are simply infeasible because they have no clue what they’re doing beyond shallow pattern-matching. [1] I imagine the same is true for o3. We simply don’t know how to make an AI that actually understands what numbers are in the way that toddlers and pigeons do.
nradov 2y ago
I think it's all (well, mostly) BS. Show me a robot that can snake out a plugged toilet.

ahartmetz 2y ago

>raw computing power grows by 10^6 every ~25 years That is really not true anymore unless there will be a new technology that can do much better than silicon electronics. Doesn't mean that were safe, but it's not that clear. Edit: to expand, a single transistor is made of a pretty pedestrian number of atoms these days. The gate pitch of TSMC "3 nm" is about 40 nm (not all of which is transistor) and silicon atoms are about 0.2 nm apart. So you have, let's say, 30^3 / 0.2^3 ~= 3 million atoms in a transistor. There is no room for another 10^6 improvement because a transistor with just a few atoms probably wouldn't work, the thermal noise would dominate it. Paraphrasing the famous Feynman quote, there isn't much room at the botton anymore, now.

shalabajser 2y ago
Your point is true but it is too narrow. A similar argument was bandied about regarding the CPU clock which hit a wall which was defeated with multi-cores and parallelization. The atom wall you are talking about is crushed with 3D semiconductors (hybrid bonding, wafer to wafer..., HBM), wafer scale CPUs like Cerebras, larger clusters, faster networking... Incidentally, massive parallelization suits AI just fine. Do you seriously think that computing power will stop growing? Ever?
- ahartmetz 2y ago
  I do think so. Every exponential growth is a sigmoid curve on a long enough timescale. The end of Moore's Law is here (for now) even with multicores and everything else, by the way. That you can get more compute by throwing more resources at the problem doesn't really count. The big deal about Moore's Law was that it was free. The 3D stuff, clusters, etc, can increase the number of transistors... but it's gonna cost proportionally more in fabrication and energy when running.

yladiz 2y ago

> raw computing power grows by 10^6 every ~25 years Good thing that the number of neurons in our brains are so many orders of magnitude larger than this that it doesn't matter. I think what you're getting at is that in the next few years we're going to have AGI, which will eventually cause societal collapse. Under that assumption, while all of the AGI arguments are pretty poor, computing power increases over time is one of the weakest. It's going to take so much more than just computing power to get us close to AGI, it's going to take a lot more than LLMs, and it's possible to poison models with specially created data, so I'm not worried; I don't have shivers. What gives me more shivers is that there are people that seriously believe the hype to the degree that they believe the current technology, just a bit improved, will lead to the end of our current life as we know it.

cheevly 2y ago
Like 80% of the human brain is used for things that AI does not need (digestion, breathing, etc). So I think that automatically disqualifies your stance, but I am open-minded and would like to know what percent of the human brain you think is vital for AI to outperform us.
- EA-3167 2y ago
  Even if you discount the role of the whole body as an integrated system, signaling and talking to itself, coordinated by the brain... you're still talking about 16 billion neurons in the cortex alone. That may sound like a number we can approach or exceed with modern tech, but neurons are not transistors. Those 16 billion neurons in the cortex form over 140 TRILLION connections through synapes. Those are both computing elements as well as memory. There's no comparison, not even close. As a bonus we don't each require a giant warehouse to house us, we don't require a vast supply of water, nor do we require a personal dedicated power plant.
  - sixstringtheory 2y ago
    Your last paragraph is the perfect rebuttal to GP. Computers don’t exist in a vacuum. They still require mineral and electrical inputs, repair, orchestration… sounds an awful lot like our bodies!

pavel_lishin 2y ago

> raw computing power grows by 10^6 every ~25 years. For how many years has this been the trend?

b112 2y ago

Raw compute power won't make AGI happen.

aithrowawaycomm 2y ago

It’s not about computing power, it’s the inadequate algorithms, and ultimately the lack of scientific knowledge. Nobody has figured out how animal brains provide the plain old common sense you’d find in a fish or a frog, nor do we have any idea how to emulate that in a machine, even though in pure bits/second a 2025 supercomputer easily outperforms a frog. It’s not just robotics or 3D navigation: Veo2 still screws up object permanence, o1-pro still screws up distinguishing small quantities[1], and 99% of AI agents are simply infeasible because they have no clue what they’re doing beyond shallow pattern-matching. [1] I imagine the same is true for o3. We simply don’t know how to make an AI that actually understands what numbers are in the way that toddlers and pigeons do.

nradov 2y ago

I think it's all (well, mostly) BS. Show me a robot that can snake out a plugged toilet.