3 ms·

Quote from article At the present rate, computers suitable for humanlike robots will appear in the 2020s. Can the pace be sustained for another three decad

Quote from article At the present rate, computers suitable for humanlike robots will appear in the 2020s. Can the pace be sustained for another three decades? The graph shows no sign of abatement. If anything, it hints that further contractions in time scale are in store. But, one often encounters thoughtful articles by knowledgeable people in the semiconductor industry giving detailed reasons why the decades of phenomenal growth must soon come to an end. I don't know where to look to confirm that can current computers do 1 billion MIPS (Million Instructions per Second?) as predicted by this article? EDIT: This Wikipedia article puts an AMD CPU at ~2 million MIPS https://en.wikipedia.org/wiki/Instructions_per_second https://en.wikipedia.org/wiki/Instructions_per_second as the highest one.

kolinko 3y ago
M1/M2s are around 2-4 trillion if I’m not mistaken
- foobiekr 3y ago
  not even within 2 orders of magnitude.
  - rerdavies 3y ago
    Just rack-mount a hundred iphones. :-P
- beautifulfreak 3y ago
  ~2.6 teraflops FP32 (CPU) for the M3 Max.
ben_w 3y ago
When the article was written, almost nobody cared about GPUs or similar architectures being really good at parallel computation. So while he wrote about MIPS, and those have indeed basically stopped significantly improving, FLOPS have continued to improve. And for AI in particular, for inference at least, we can get away with 8 bit floats, which is why my phone does 1.58e13/second, only a factor of x60 from a million-billion.
pulvinar 3y ago
Today we'd include the GPU. If we include supercomputers, we really have to compare total global capacity, which is on the order of 1 million times that of 1997 [0]. That basically fits his trend line. [0]: https://ourworldindata.org/grapher/supercomputer-power-flops https://ourworldindata.org/grapher/supercomputer-power-flops
- geysersam 3y ago
  The graph in the article showing computing power per $1000, doesn't that make the comparison to supercomputers misleading?
floxy 3y ago
Nvidia claims the H100 GPU can do >3e15 8-bit operations per second (3 billion - million instructions per second): https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor-core-gpu-datasheet https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor...
ertgbnm 3y ago
The promise of Instruction per Second scaling forever did stop being super-exponential. Instead of CPU based Instructions per Second we have continued scaling compute with GPU based Floating-point Operations Per Second which don't exactly have a 1:1 equivalence with instructions but are close enough. Nvidia released the "first" GPU in 1999 so it's not too surprising that this guy wasn't thinking in those terms in 1998. That's why Kurzweil and Bostrom used FLOPS in their extrapolations. They also end up in the neighborhood of 2020s/2030s in their later predictions in the 2000s. With that in mind we are currently at 1 billion GigaFLOPS for the largest super computer in 2023. And the trendline since the 90's has kept up with the prediction made in the post. Although we can't exactly say 1 GigaFLOPS = 1000 MIPS. https://ourworldindata.org/grapher/supercomputer-power-flops?time=earliest..2023 https://ourworldindata.org/grapher/supercomputer-power-flops...
jetrink 3y ago
> I don't know where to look to confirm that can current computers do 1 billion MIPS (Million Instructions per Second?) as predicted by this article? An RTX 4090 has AI compute of up to 661 teraflops in FP16[1]. FLOPS and IPS are not interchangeable, but just for fun, that's 6.6 * 10^14 FLOPS, which is just short of 1 billion million (10^15). 1. https://www.tomshardware.com/features/nvidia-ada-lovelace-and-geforce-rtx-40-series-everything-we-know https://www.tomshardware.com/features/nvidia-ada-lovelace-an...
tim333 3y ago
I'm not sure he predicts a billion mips. The graph tops out at a million mips per $1000. He figues brain equivalence is 100 million mips. If you assume mips are equivalent to flops then that's 100 teraflops. That would seem roughly to match: >"NVIDIA GeForce RTX 4090 Is The First Gaming Graphics Card To Deliver 100 TFLOPs of Compute Performance" https://wccftech.com/nvidia-geforce-rtx-4090-is-the-first-gaming-graphics-card-to-deliver-100-tflops-of-compute-performance/ https://wccftech.com/nvidia-geforce-rtx-4090-is-the-first-ga... Currently about $1,800 on Amazon.

kolinko 3y ago

M1/M2s are around 2-4 trillion if I’m not mistaken

foobiekr 3y ago
not even within 2 orders of magnitude.
- rerdavies 3y ago
  Just rack-mount a hundred iphones. :-P
beautifulfreak 3y ago
~2.6 teraflops FP32 (CPU) for the M3 Max.

ben_w 3y ago

When the article was written, almost nobody cared about GPUs or similar architectures being really good at parallel computation. So while he wrote about MIPS, and those have indeed basically stopped significantly improving, FLOPS have continued to improve. And for AI in particular, for inference at least, we can get away with 8 bit floats, which is why my phone does 1.58e13/second, only a factor of x60 from a million-billion.

pulvinar 3y ago

Today we'd include the GPU. If we include supercomputers, we really have to compare total global capacity, which is on the order of 1 million times that of 1997 [0]. That basically fits his trend line. [0]: https://ourworldindata.org/grapher/supercomputer-power-flops https://ourworldindata.org/grapher/supercomputer-power-flops

geysersam 3y ago
The graph in the article showing computing power per $1000, doesn't that make the comparison to supercomputers misleading?

floxy 3y ago

Nvidia claims the H100 GPU can do >3e15 8-bit operations per second (3 billion - million instructions per second): https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor-core-gpu-datasheet https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor...

ertgbnm 3y ago

The promise of Instruction per Second scaling forever did stop being super-exponential. Instead of CPU based Instructions per Second we have continued scaling compute with GPU based Floating-point Operations Per Second which don't exactly have a 1:1 equivalence with instructions but are close enough. Nvidia released the "first" GPU in 1999 so it's not too surprising that this guy wasn't thinking in those terms in 1998. That's why Kurzweil and Bostrom used FLOPS in their extrapolations. They also end up in the neighborhood of 2020s/2030s in their later predictions in the 2000s. With that in mind we are currently at 1 billion GigaFLOPS for the largest super computer in 2023. And the trendline since the 90's has kept up with the prediction made in the post. Although we can't exactly say 1 GigaFLOPS = 1000 MIPS. https://ourworldindata.org/grapher/supercomputer-power-flops?time=earliest..2023 https://ourworldindata.org/grapher/supercomputer-power-flops...

jetrink 3y ago

> I don't know where to look to confirm that can current computers do 1 billion MIPS (Million Instructions per Second?) as predicted by this article? An RTX 4090 has AI compute of up to 661 teraflops in FP16[1]. FLOPS and IPS are not interchangeable, but just for fun, that's 6.6 * 10^14 FLOPS, which is just short of 1 billion million (10^15). 1. https://www.tomshardware.com/features/nvidia-ada-lovelace-and-geforce-rtx-40-series-everything-we-know https://www.tomshardware.com/features/nvidia-ada-lovelace-an...

tim333 3y ago

I'm not sure he predicts a billion mips. The graph tops out at a million mips per $1000. He figues brain equivalence is 100 million mips. If you assume mips are equivalent to flops then that's 100 teraflops. That would seem roughly to match: >"NVIDIA GeForce RTX 4090 Is The First Gaming Graphics Card To Deliver 100 TFLOPs of Compute Performance" https://wccftech.com/nvidia-geforce-rtx-4090-is-the-first-gaming-graphics-card-to-deliver-100-tflops-of-compute-performance/ https://wccftech.com/nvidia-geforce-rtx-4090-is-the-first-ga... Currently about $1,800 on Amazon.