5 ms·

2008? So... anything happen since? I assume nobody took him up on his offer! "Alan Kay: [...] "Just as an aside [...] a benchmark from 1979 at Xerox PARC runs

by _chris_ 11y ago

2008? So... anything happen since? I assume nobody took him up on his offer! "Alan Kay: [...] "Just as an aside [...] a benchmark from 1979 at Xerox PARC runs only 50 times faster today. Moore’s law has given us somewhere between 40,000 and 60,000 times improvement in that time. So there’s approximately a factor of 1,000 in efficiency that has been lost by bad CPU architectures."" Is he complaining that machines today aren't a bunch of micro-coded, multi-chip computers like in the good old days, or does he have a better proposal to the modern multi-core, superscalar OoO processors coupled to GPGPU multi-threaded engines? Because complaining about Moore's Law feels a bit meaningless if DRAM is still 100ns away.

_yosefk 11y ago
In 2015 I find the claim that you could beat Intel by 1000x with a better arch much more idiotic than I did in 2008. Because apart from it being idiotic from a technical standpoint I'm now dumbfounded by the extent of its idiocy from the business angle. If you're so smart, what's stopping you from making a trillion dollars off it? You need to be an Alan Kay to utter such bullshit repeatedly and still have crowds of worshippers taking each of your utterances as gospel while chanting that you singlehandedly invented computing as we know it.
- creshal 11y ago
  You'd think there are enough universities working on research processor architectures that a 10x would have popped up by now, if you can theoretically make a 1000x one. I'd rather point the blame on software: If we were banging on hardware directly like in the 70s, we'd probably see a massive performance boost, too… just hope that your customers accidentally happen to have the hardware you optimized for and don't want to run anything else on it.
- CyberDildonics 11y ago
  I don't know if 1000x is something that can be backed up, but the difference between a program written with poor memory locality and one written with great locality plus 256 bit SIMD can give a performance boost of around 100x. This isn't theoretical, I've done it personally and this is just one core. So the claims are anything but outrageous although that is basically what GPUs are - transistors going to actual FLOPS instead of dealing with memory latency. Intel cores however are structured like this because most code isn't written with cache locality in mind, they are written in god awful pointer chasing spaghetti styles so all the work that Intel puts in to make that go fast pays off. Skylake almost had SIMD scatter and gather, which could have been an enormous boost to properly written software, but they had no reason to put it in when they are curb stomping AMD on one hand, and very little software takes advantage of the SIMD units they have now on the other.
- sklogic 11y ago
  It's trivial to beat a generic CPU by 1000x with a specialised ASIC or even an FPGA in many cases (think cryptography, for example).
  - adwn 11y ago
    Make that a few cases for FPGAs. Modern desktop CPUs are incredibly fast and run circles around modern FPGAs, even for many specialized, repetitive, parallel, integer-computation-bound algorithms. Of course, there are a couple of applications where FPGAs are actually faster than CPUs. Source: I'm an FPGA developer with a software engineering background.
    - sklogic 11y ago
      > Modern desktop CPUs are incredibly fast and run circles around modern FPGAs Yep. At a tiny price of tenths of W of TDP. FPGAs shine in communication, not in computing. Fast isolated memories, wide channels (as wide as you like), etc. And there are far more tasks where you have to juggle gigabytes of data as fast as possible than the tasks where you have to maximise FLOPS.
    - kragen 11y ago
      That's fascinating and startling! Why is that? Is it that the FPGA's cost per bit operation is just so many times larger than the CPU's that the FPGA can't compete? Where do GPUs fit in?
      - adwn 11y ago
        Several factors: 1) FPGAs are much slower due to the overhead of the flexibility they provide. With a lot of effort, you can get some FPGA designs (not even all of them!) to maybe 400 MHz on the more expensive FPGAs, and then it will do much less per cycle compared to an ASIC (CPUs are ASICs), so you need deep pipelines, with all the cost, complexity, and restrictions this entails. 2) CPUs have a lot of memory on-chip in the form of cache (several MiB), and they have sophisticated prefetch logic. In FPGAs, memory comes in several 100 blocks of, say, 4 kiB (depending on FPGA model). All memory management is purely manual. 3) CPUs tend to have a much higher external memory bandwidth. 4) Complex operations that are not directly provided by the FPGA fabric, like floating-point arithmetic or large memories with more than two ports, are expensive in terms of area and performance. 5) Modern desktop CPUs are incredibly well optimized: high clock frequency, out-of-order execution, superscalarity with several execution units, SIMD, automatic cache management, branch prediction, ... It's just very hard to beat that, and this is only possible for certain, very restricted applications. Highly parallel DSP (Digital Signal Processing) with integer/fixed-point arithmetic comes to mind.
- stevejones 11y ago
  You'd be better off working out a way to compile smalltalk to FPGA.
mseebach 11y ago
What exactly is the benchmark? Is it, say, pure CPU-bound maths? Because a lot of the work between 1979 and today has been about getting the right numbers to the CPU faster, not (just) for the CPU to actually do the math on the numbers. One of the biggest leaps in general computing performance in recent years have been the general availability of SSDs. SSDs makes you computer significantly faster, but not by changing anything about the CPU.
venomsnake 11y ago
I would kindly show Mr. Kay god of war 2 on the original PS2. Which is incredible tribute to what can be squeezed out of modest hardware. We have glaring inefficiencies in every level of the software development and runtime stack. I have been writing software for 20 years. The sluggishness of the software, difficulty of writing and debugging has remained constant. Creating a modern Boostrap singlepage app is as difficult and time consuming as the old MFC 4.0 stack. Similar speed of delivery too. Hardware is fast enough. We just don't bother using it.
kragen 11y ago
I dug into the 50× claim in https://www.mail-archive.com/kragen-tol@canonical.org/msg00185.html https://www.mail-archive.com/kragen-tol@canonical.org/msg001.... Basically Alan was saying that the all-ECL Dorado, which was never sold commercially but might have cost US$100k per computer if it had been, was 50× slower than modern (2005) embedded RISC processors that cost US$150 per computer. Omitting the three orders of magnitude in performance due to sheer brute cost is how you get to "only 50 times faster today". There's an additional order of magnitude of overhead in Squeak's bytecode-interpretation strategy, which was a good strategy on the Dorado but not on an AMD64; you can squeeze that out with JIT, and you get to four orders of magnitude discrepancy between Kay's offhand claim and reality. If you build a 500-node cluster and use a modern JIT compiler on it, the "efficiency that has been lost by bad CPU architectures" evaporates. As you implicitly point out, though, this is parallel performance, not serial performance. Kay's group has done a substantial amount of awesome research since he said that in 2007, dramatically exceeding Kay's expectations for performance on modern machines. To a great extent, Kay's Communication Design Group (funded by VPRI and SAP) is the closest thing we have today to the 1970s Xerox PARC, pioneering new enabling technologies and interaction paradigms.
- hga 11y ago
  Also worth pointing out is that ECL processors were specialized hot rods, not in the main line of contemporary computer systems aside from supercomputers and the very high end of mainframes like the top of the line DEC KL10 PDP-10 CPU, Vax 8000 family, and IBM 360/91. They were expensive to design and run (traded power dissipation for speed), and eventually CMOS caught up with ECL.
  - kragen 11y ago
    Yes, that's very important context to have. I should have explained that. Thank you.

by _chris_ 11y ago

_yosefk 11y ago

In 2015 I find the claim that you could beat Intel by 1000x with a better arch much more idiotic than I did in 2008. Because apart from it being idiotic from a technical standpoint I'm now dumbfounded by the extent of its idiocy from the business angle. If you're so smart, what's stopping you from making a trillion dollars off it? You need to be an Alan Kay to utter such bullshit repeatedly and still have crowds of worshippers taking each of your utterances as gospel while chanting that you singlehandedly invented computing as we know it.

creshal 11y ago
You'd think there are enough universities working on research processor architectures that a 10x would have popped up by now, if you can theoretically make a 1000x one. I'd rather point the blame on software: If we were banging on hardware directly like in the 70s, we'd probably see a massive performance boost, too… just hope that your customers accidentally happen to have the hardware you optimized for and don't want to run anything else on it.
CyberDildonics 11y ago
I don't know if 1000x is something that can be backed up, but the difference between a program written with poor memory locality and one written with great locality plus 256 bit SIMD can give a performance boost of around 100x. This isn't theoretical, I've done it personally and this is just one core. So the claims are anything but outrageous although that is basically what GPUs are - transistors going to actual FLOPS instead of dealing with memory latency. Intel cores however are structured like this because most code isn't written with cache locality in mind, they are written in god awful pointer chasing spaghetti styles so all the work that Intel puts in to make that go fast pays off. Skylake almost had SIMD scatter and gather, which could have been an enormous boost to properly written software, but they had no reason to put it in when they are curb stomping AMD on one hand, and very little software takes advantage of the SIMD units they have now on the other.
sklogic 11y ago
It's trivial to beat a generic CPU by 1000x with a specialised ASIC or even an FPGA in many cases (think cryptography, for example).
- adwn 11y ago
  Make that a few cases for FPGAs. Modern desktop CPUs are incredibly fast and run circles around modern FPGAs, even for many specialized, repetitive, parallel, integer-computation-bound algorithms. Of course, there are a couple of applications where FPGAs are actually faster than CPUs. Source: I'm an FPGA developer with a software engineering background.
  - sklogic 11y ago
    > Modern desktop CPUs are incredibly fast and run circles around modern FPGAs Yep. At a tiny price of tenths of W of TDP. FPGAs shine in communication, not in computing. Fast isolated memories, wide channels (as wide as you like), etc. And there are far more tasks where you have to juggle gigabytes of data as fast as possible than the tasks where you have to maximise FLOPS.
  - kragen 11y ago
    That's fascinating and startling! Why is that? Is it that the FPGA's cost per bit operation is just so many times larger than the CPU's that the FPGA can't compete? Where do GPUs fit in?
    - adwn 11y ago
      Several factors: 1) FPGAs are much slower due to the overhead of the flexibility they provide. With a lot of effort, you can get some FPGA designs (not even all of them!) to maybe 400 MHz on the more expensive FPGAs, and then it will do much less per cycle compared to an ASIC (CPUs are ASICs), so you need deep pipelines, with all the cost, complexity, and restrictions this entails. 2) CPUs have a lot of memory on-chip in the form of cache (several MiB), and they have sophisticated prefetch logic. In FPGAs, memory comes in several 100 blocks of, say, 4 kiB (depending on FPGA model). All memory management is purely manual. 3) CPUs tend to have a much higher external memory bandwidth. 4) Complex operations that are not directly provided by the FPGA fabric, like floating-point arithmetic or large memories with more than two ports, are expensive in terms of area and performance. 5) Modern desktop CPUs are incredibly well optimized: high clock frequency, out-of-order execution, superscalarity with several execution units, SIMD, automatic cache management, branch prediction, ... It's just very hard to beat that, and this is only possible for certain, very restricted applications. Highly parallel DSP (Digital Signal Processing) with integer/fixed-point arithmetic comes to mind.
stevejones 11y ago
You'd be better off working out a way to compile smalltalk to FPGA.

mseebach 11y ago

What exactly is the benchmark? Is it, say, pure CPU-bound maths? Because a lot of the work between 1979 and today has been about getting the right numbers to the CPU faster, not (just) for the CPU to actually do the math on the numbers. One of the biggest leaps in general computing performance in recent years have been the general availability of SSDs. SSDs makes you computer significantly faster, but not by changing anything about the CPU.

venomsnake 11y ago

I would kindly show Mr. Kay god of war 2 on the original PS2. Which is incredible tribute to what can be squeezed out of modest hardware. We have glaring inefficiencies in every level of the software development and runtime stack. I have been writing software for 20 years. The sluggishness of the software, difficulty of writing and debugging has remained constant. Creating a modern Boostrap singlepage app is as difficult and time consuming as the old MFC 4.0 stack. Similar speed of delivery too. Hardware is fast enough. We just don't bother using it.

kragen 11y ago

I dug into the 50× claim in https://www.mail-archive.com/kragen-tol@canonical.org/msg00185.html https://www.mail-archive.com/kragen-tol@canonical.org/msg001.... Basically Alan was saying that the all-ECL Dorado, which was never sold commercially but might have cost US$100k per computer if it had been, was 50× slower than modern (2005) embedded RISC processors that cost US$150 per computer. Omitting the three orders of magnitude in performance due to sheer brute cost is how you get to "only 50 times faster today". There's an additional order of magnitude of overhead in Squeak's bytecode-interpretation strategy, which was a good strategy on the Dorado but not on an AMD64; you can squeeze that out with JIT, and you get to four orders of magnitude discrepancy between Kay's offhand claim and reality. If you build a 500-node cluster and use a modern JIT compiler on it, the "efficiency that has been lost by bad CPU architectures" evaporates. As you implicitly point out, though, this is parallel performance, not serial performance. Kay's group has done a substantial amount of awesome research since he said that in 2007, dramatically exceeding Kay's expectations for performance on modern machines. To a great extent, Kay's Communication Design Group (funded by VPRI and SAP) is the closest thing we have today to the 1970s Xerox PARC, pioneering new enabling technologies and interaction paradigms.

hga 11y ago
Also worth pointing out is that ECL processors were specialized hot rods, not in the main line of contemporary computer systems aside from supercomputers and the very high end of mainframes like the top of the line DEC KL10 PDP-10 CPU, Vax 8000 family, and IBM 360/91. They were expensive to design and run (traded power dissipation for speed), and eventually CMOS caught up with ECL.
- kragen 11y ago
  Yes, that's very important context to have. I should have explained that. Thank you.