8 ms·

I’ve made similar comments in the past but I think we’re just trying to predict too much about what programs are trying to do in hardware. I’d rather have simp

I’ve made similar comments in the past but I think we’re just trying to predict too much about what programs are trying to do in hardware. I’d rather have simple hardware that is light on energy requirements and easier to understand. I don’t think software as an industry really has a “this chip isn’t fast enough problem”. Most of the real slowdowns anyone has in day to day performance has more to do with inefficient code than hardware anyways. Look at old game consoles (like a regular old PlayStation) and what they were able to render on what today would be laughably slow hardware. They could do it because developers learned the stack inside and out. I’m hopeful that the emergence of arm as a desktop/server competitor will help in some ways by putting enough pressure on companies like intel to really rethink some of their core stuff, but who knows.

mkr-hn 5y ago
See: racing the beam https://www.youtube.com/watch?v=sJFnWZH5FXc https://www.youtube.com/watch?v=sJFnWZH5FXc The things they did with 128 bytes of RAM were amazing.
UncleMeat 5y ago
> I’d rather have simple hardware that is light on energy requirements and easier to understand. I don’t think software as an industry really has a “this chip isn’t fast enough problem”. Turning off speculative execution reduces performance enormously. Yes, code is often less efficient than it could be but "surprise, you need 5x as large of a datacenter because your hardware isn't doing fancy stuff" is not going to be an acceptable thing for any large organization.
- icegreentea2 5y ago
  More specifically, we should point out that speculative execution is important for papering over IO/memory latency. We can incant all we want about 'inefficient code', but the compute vs memory latency imbalance is a whole different beast.
- infogulch 5y ago
  I suspect our heavy reliance on speculative execution is just a local optimum, and we can grow out of it by providing the processor more information about the data flow, making memory access more explicitly asynchronous, and simplifying the hot paths. I like the way the mill approaches these architectural problems.
  - zozbot234 5y ago
    The interesting part about the Mill is that it's not even all that complex. It's just taking the existing VLIW architectural approach and adding a number of custom "tweaks" to improve the programming model over what VLIW provides.
  - tux3 5y ago
    I'm very unconvinced by these arguments. Providing enough information to the processor to claw back all the performance that predictors give you, that requires some way of knowing all those things in advance. Statically. And so you fall into the pit of tar and despair that is relying on Sufficiently Smart Compilers. Same one that couldn't save Intel's shiny new IA-64 architecture (the "Itanic"). Static analysis is just not a tractable way to replace hardware predictors. Even profile-guided information is a questionable answer. Could it be we're missing some key idea that'll take us far beyond all of this? Maybe, but it's probably not the same WLIV designs that require impossible feats of software comparable to asking for the weather 2 years in advance so you can save yourself the hardware cost of an umbrella. Now, perhaps it would be unfair to criticize the Mill for not taping out an actual chip, after all having new ideas is valuable and fabs are very expensive. But we know this is a tricky subject. Performance optimization ideas only have value after you benchmark them, and they don't have an RTL implementation. Nothing that can run on an FPGA, not even "abstract" Verilog running in a testbench on a software simulator. If what you want is a DSP that runs super simple hot loops very fast, then build a DSP and make it VLIW all you like. But that's not the workloads people run on a general purpose computer.
    - wizzwizz4 5y ago
      > And so you fall into the pit of tar and despair that is relying on Sufficiently Smart Compilers. How will we ever know that a Sufficiently Smart Compiler is impossible, if we never have processors where compiler intelligence is useful?
josh2600 5y ago
We’re seeing a full frontal assault on Intel by competing computing architectures. Everyone is jockeying to see who is going to have the better next gen architecture. The problem is that in order to start attacking general computing as a problem, you need first customers who buy a lot of chips, and those customers tend to have specific workloads they want accelerated. Now we’re back to designing for customers instead of abstractly making the absolute best chip. And that’s the reality right? Demand drives innovation it private enterprise is funding it. If you want a chip for the public good, you’d need to fund it with public money and that’s a different dance with the devil. In short, I think the way that technology is produced is fundamentally customer focused right now.
- im3w1l 5y ago
  I think this has cause and effect reversed. Advancing general purpose CPU's is very hard and expensive nowadays. The slower rate of progress means that custom chips make more sense, they take longer to become obsolete.
astrange 5y ago
> I’d rather have simple hardware that is light on energy requirements and easier to understand. I don’t think software as an industry really has a “this chip isn’t fast enough problem”. Have you seen how happy people are with M1 machines? That's because it's faster. It's definitely not simpler. > Most of the real slowdowns anyone has in day to day performance has more to do with inefficient code than hardware anyways. Not true, especially not for heat. The energy use of on-CPU/on-GPU tasks is more up to the hardware. It might be true for things like network bandwidth though.
- saagarjha 5y ago
  Wait until Slack outfits their developers with M1 machines and we’ll quickly be back to being unhappy again.
  - josteink 5y ago
    I know this sounds radical, but uh, if everyone hates slack how about not using slack?
    - saagarjha 5y ago
      This is often the same as saying "how about you quit your job".
      - josteink 5y ago
        But why does companies use slack if “everyone” hates it? For once, I’ve never been to a company which uses it. There are literally countless options in the IM/groupware field which is not Slack.
        mhuffman 5y ago
        The only two I have seen in most places are Slack and Teams.
temac 5y ago
You would be surprised how "slow" a non speculative processor is. Even Intel Atoms have been speculative since Silvermont. If you take more mainstream historical processors, for example: https://www.cpu-world.com/Compare/385/Intel_Pentium_Dual-Core_G6950_vs_Intel_Pentium_MMX_166_MHz_(A80503166).html https://www.cpu-world.com/Compare/385/Intel_Pentium_Dual-Cor... Let say the Pentium MMX 166 is roughly 60x slower, per core, than the Pentium G6950. So the "IPC" (I'm not sure of what the benchmark are doing, but I'm doing rough evaluations anyway) is approx 3.5 lower for the Pentium MMX. And that is while the speed gap constantly increased between memory and CPU (esp latency), so a Pentium MMX cpu core somehow speed up to 2.8 GHz would actually probably perform even worse in a design with memory that actually exists. Maybe way worse. The complexity of modern good ARMs is comparable or even higher in some area than the complexity of x86 processors.
mhh__ 5y ago
But rendering isn't what makes a modern computer slow. For example, Assassin's creed isn't even that well optimized but it maxes all threads, uses all the GPU and looks beautiful. And people have tried "simple" hardware (it ends up being not simple) that the programmer (compiler) understands before, it doesn't work.
paulryanrogers 5y ago
Consider the philosophy of MAME which plays old games and avoids optimizing for GPUs and (IIRC) most specialized graphics layers like Direct3D or Metal. As I understand it this makes it more portable and easier to maintain, at the cost of some performance. Now MAME's scope is limited to aging games which aren't evolving. Only the underlyibg OS's and runtime hardware are changing. Console games pushing aging hardware have a very small variety of runtime hardware to optimize for. And optimizations pay off in better graphics or features that differentiate. Optimizing for ever evolving desktop computers with near infinite hardware is a whole different story.
jcelerier 5y ago
> I don’t think software as an industry really has a “this chip isn’t fast enough problem”. yet this is what I'm saying to myself every day, even with very fast computers available, and very efficient code running that hardware pretty much to its limits.
giantrobot 5y ago
> Look at old game consoles (like a regular old PlayStation) and what they were able to render on what today would be laughably slow hardware. They could do it because developers learned the stack inside and out. You're oversimplifying the situation and drawing conclusions from it. The PSX had dedicated hardware for geometry transforms, video decoding, sound processing, and rasterizing. The CPU was used for game logic and queuing up the render pipeline. So it's not like all PlayStation developers were doing black magic in every game, they were using a lot of dedicated hardware with well known and relatively modest and fixed constraints (NTSC/PAL, stereo audio, and relatively small frame buffers). It's interesting you picked the PSX as your exemplar because it's CPU/GPU/MDEC and audio processor were meant to be black boxes so developers didn't have to code to bare metal. The IO stack was much shorter and the OS (such as it was) was single tasking, single threaded, and didn't have any networking. Slowdowns in modern systems are much more complicated that a simplistic "inefficient code makes them slow". There's hundreds of small stupid latencies all over the place in modern systems. The USB and Bluetooth stacks are filled with little latencies, retransmissions, and just wait loops. Unless your inefficient code is full of accidentally quadratic loops it's usually compounded latencies making systems "feel slow".
- p_l 5y ago
  The description sounds more like PS4 than PS2 (maybe PSX was simpler - PS3 is closer to PS4 and "normal" programming). I've seen numerous examples of how involved CPU was in rendering, especially on PS2, due to special units being exposed as MIPS coprocessors. Efficient rendering meant you had to keep the pipelines happy and that could require significant amount of assembly wrangling. I remember reading about how GTA3 on PS2 was a watershed moment, because it was a proof you could use a somewhat more generic engine - whereas console programming had a heavy amount of assembly focus due to historical reasons.
  - giantrobot 5y ago
    I'm sure there's plenty of people on HN with first hand experience developing for the various PlayStation models. I only know what I've read about the experience and technical tear downs. So maybe one will swoop in dropping knowledge bombs and tell me I'm wrong. The original claim was 3D worked on the PSX because of "efficiency" of the code. While I'm not claiming games on the PSX were inefficient, the hardware was designed specifically to do real-time 3D rendering. It's not like the PSX could only do 3D because everyone somehow had super deep mastery of the system. To my understanding the PSX was a big change in console development. While consoles always had some dedicated hardware (blitters, audio DSPs, etc), development was a lot of beam racing. There were no frame buffers, code ran as graphics were being drawn to the screen and during the VBI. The PSX was more like a modern system where you had a CPU and a GPU (albeit fixed function) with an OpenGL-ish drawing model. It was a far more capable 3D machine than PCs of the time that had to do all 3D work on the CPU with no acceleration. Tomb Raider on the PSX was impressive because the console only cost $300. While I'm sure Tomb Raider on the PSX wasn't inefficiently coded, the design of the PSX console contributed as much to its existence as the code. Quake on the PC on the other hand could only exist because of the developers' deep system mastery pushing generic hardware to its limits (before GPUs were prevalent).
quotemstr 5y ago
I'll make a deal with you: I'll agree to turn down speculative execution in hardware if 1) you can solve the VLIW scheduling problem, and 2) you transition the entire computing ecosystem to a JIT model. These are the things we would need to claw back the performance we would lose through disabling speculation. You can look at speculation (I'm handwaving a lot here, bear with me) as the processor hardware dynamically recompiling your program code depending on observed behavior of the program. That's well and good and it gets us a huge performance boost. You can, in principle, do the same thing in pure software. But every single attempt to do has ended in total algorithmic failure. The last serious attempt I'm aware of to "drive" a uOP scheduler explicitly in software was Itanium, and that failed, in part, because compilers couldn't take advantage of the processor's instruction level parallelism. There's nothing in math or mathematics or computer science that forbids a magical compiler of the sort the Itanium people wanted to create. But nobody's made one. Your first task in your project of eliminating speculative execution is to solve this algorithmic problem. But solving problem #1, while necessary, is insufficient. No static ahead-of-time compiler can adjust the compiled code depending on the actual execution history of the program. To really get back to speculative execution par, you have to give your already-magical compiler the ability to recompile code at runtime. That means turning everything into a JIT. Your /bin/ls would actually be LLVM bytecode, not machine code, and some runtime system would be responsible for dynamically generating the machine code and adjusting it depending on execution history. After all, that's what current superscalar CPUs do internally, transparently, all the time. This is problem #2. Honestly, I think the world we'd create by solving both these problems would be a better world. I really don't like how we can't program the processor's speculation engine and uOP scheduler. I'd love to be able to do that. But I don't think we can get there from where we are, so we're going to be stuck with speculation and hardware mitigation forever. Please, prove me wrong.
- rmdashrfstar 5y ago
  > Honestly, I think the world we'd create by solving both these problems would be a better world. I really don't like how we can't program the processor's speculation engine and uOP scheduler. I'd love to be able to do that. Have you started working on any solutions for 1 and 2?

mkr-hn 5y ago

See: racing the beam https://www.youtube.com/watch?v=sJFnWZH5FXc https://www.youtube.com/watch?v=sJFnWZH5FXc The things they did with 128 bytes of RAM were amazing.

UncleMeat 5y ago

> I’d rather have simple hardware that is light on energy requirements and easier to understand. I don’t think software as an industry really has a “this chip isn’t fast enough problem”. Turning off speculative execution reduces performance enormously. Yes, code is often less efficient than it could be but "surprise, you need 5x as large of a datacenter because your hardware isn't doing fancy stuff" is not going to be an acceptable thing for any large organization.

icegreentea2 5y ago
More specifically, we should point out that speculative execution is important for papering over IO/memory latency. We can incant all we want about 'inefficient code', but the compute vs memory latency imbalance is a whole different beast.
infogulch 5y ago
I suspect our heavy reliance on speculative execution is just a local optimum, and we can grow out of it by providing the processor more information about the data flow, making memory access more explicitly asynchronous, and simplifying the hot paths. I like the way the mill approaches these architectural problems.
- zozbot234 5y ago
  The interesting part about the Mill is that it's not even all that complex. It's just taking the existing VLIW architectural approach and adding a number of custom "tweaks" to improve the programming model over what VLIW provides.
- tux3 5y ago
  I'm very unconvinced by these arguments. Providing enough information to the processor to claw back all the performance that predictors give you, that requires some way of knowing all those things in advance. Statically. And so you fall into the pit of tar and despair that is relying on Sufficiently Smart Compilers. Same one that couldn't save Intel's shiny new IA-64 architecture (the "Itanic"). Static analysis is just not a tractable way to replace hardware predictors. Even profile-guided information is a questionable answer. Could it be we're missing some key idea that'll take us far beyond all of this? Maybe, but it's probably not the same WLIV designs that require impossible feats of software comparable to asking for the weather 2 years in advance so you can save yourself the hardware cost of an umbrella. Now, perhaps it would be unfair to criticize the Mill for not taping out an actual chip, after all having new ideas is valuable and fabs are very expensive. But we know this is a tricky subject. Performance optimization ideas only have value after you benchmark them, and they don't have an RTL implementation. Nothing that can run on an FPGA, not even "abstract" Verilog running in a testbench on a software simulator. If what you want is a DSP that runs super simple hot loops very fast, then build a DSP and make it VLIW all you like. But that's not the workloads people run on a general purpose computer.
  - wizzwizz4 5y ago
    > And so you fall into the pit of tar and despair that is relying on Sufficiently Smart Compilers. How will we ever know that a Sufficiently Smart Compiler is impossible, if we never have processors where compiler intelligence is useful?

josh2600 5y ago

We’re seeing a full frontal assault on Intel by competing computing architectures. Everyone is jockeying to see who is going to have the better next gen architecture. The problem is that in order to start attacking general computing as a problem, you need first customers who buy a lot of chips, and those customers tend to have specific workloads they want accelerated. Now we’re back to designing for customers instead of abstractly making the absolute best chip. And that’s the reality right? Demand drives innovation it private enterprise is funding it. If you want a chip for the public good, you’d need to fund it with public money and that’s a different dance with the devil. In short, I think the way that technology is produced is fundamentally customer focused right now.

im3w1l 5y ago
I think this has cause and effect reversed. Advancing general purpose CPU's is very hard and expensive nowadays. The slower rate of progress means that custom chips make more sense, they take longer to become obsolete.

astrange 5y ago

> I’d rather have simple hardware that is light on energy requirements and easier to understand. I don’t think software as an industry really has a “this chip isn’t fast enough problem”. Have you seen how happy people are with M1 machines? That's because it's faster. It's definitely not simpler. > Most of the real slowdowns anyone has in day to day performance has more to do with inefficient code than hardware anyways. Not true, especially not for heat. The energy use of on-CPU/on-GPU tasks is more up to the hardware. It might be true for things like network bandwidth though.

saagarjha 5y ago
Wait until Slack outfits their developers with M1 machines and we’ll quickly be back to being unhappy again.
- josteink 5y ago
  I know this sounds radical, but uh, if everyone hates slack how about not using slack?
  - saagarjha 5y ago
    This is often the same as saying "how about you quit your job".
    - josteink 5y ago
      But why does companies use slack if “everyone” hates it? For once, I’ve never been to a company which uses it. There are literally countless options in the IM/groupware field which is not Slack.
      - mhuffman 5y ago
        The only two I have seen in most places are Slack and Teams.

temac 5y ago

You would be surprised how "slow" a non speculative processor is. Even Intel Atoms have been speculative since Silvermont. If you take more mainstream historical processors, for example: https://www.cpu-world.com/Compare/385/Intel_Pentium_Dual-Core_G6950_vs_Intel_Pentium_MMX_166_MHz_(A80503166).html https://www.cpu-world.com/Compare/385/Intel_Pentium_Dual-Cor... Let say the Pentium MMX 166 is roughly 60x slower, per core, than the Pentium G6950. So the "IPC" (I'm not sure of what the benchmark are doing, but I'm doing rough evaluations anyway) is approx 3.5 lower for the Pentium MMX. And that is while the speed gap constantly increased between memory and CPU (esp latency), so a Pentium MMX cpu core somehow speed up to 2.8 GHz would actually probably perform even worse in a design with memory that actually exists. Maybe way worse. The complexity of modern good ARMs is comparable or even higher in some area than the complexity of x86 processors.

mhh__ 5y ago

But rendering isn't what makes a modern computer slow. For example, Assassin's creed isn't even that well optimized but it maxes all threads, uses all the GPU and looks beautiful. And people have tried "simple" hardware (it ends up being not simple) that the programmer (compiler) understands before, it doesn't work.

paulryanrogers 5y ago

Consider the philosophy of MAME which plays old games and avoids optimizing for GPUs and (IIRC) most specialized graphics layers like Direct3D or Metal. As I understand it this makes it more portable and easier to maintain, at the cost of some performance. Now MAME's scope is limited to aging games which aren't evolving. Only the underlyibg OS's and runtime hardware are changing. Console games pushing aging hardware have a very small variety of runtime hardware to optimize for. And optimizations pay off in better graphics or features that differentiate. Optimizing for ever evolving desktop computers with near infinite hardware is a whole different story.

jcelerier 5y ago

> I don’t think software as an industry really has a “this chip isn’t fast enough problem”. yet this is what I'm saying to myself every day, even with very fast computers available, and very efficient code running that hardware pretty much to its limits.

giantrobot 5y ago

> Look at old game consoles (like a regular old PlayStation) and what they were able to render on what today would be laughably slow hardware. They could do it because developers learned the stack inside and out. You're oversimplifying the situation and drawing conclusions from it. The PSX had dedicated hardware for geometry transforms, video decoding, sound processing, and rasterizing. The CPU was used for game logic and queuing up the render pipeline. So it's not like all PlayStation developers were doing black magic in every game, they were using a lot of dedicated hardware with well known and relatively modest and fixed constraints (NTSC/PAL, stereo audio, and relatively small frame buffers). It's interesting you picked the PSX as your exemplar because it's CPU/GPU/MDEC and audio processor were meant to be black boxes so developers didn't have to code to bare metal. The IO stack was much shorter and the OS (such as it was) was single tasking, single threaded, and didn't have any networking. Slowdowns in modern systems are much more complicated that a simplistic "inefficient code makes them slow". There's hundreds of small stupid latencies all over the place in modern systems. The USB and Bluetooth stacks are filled with little latencies, retransmissions, and just wait loops. Unless your inefficient code is full of accidentally quadratic loops it's usually compounded latencies making systems "feel slow".

p_l 5y ago
The description sounds more like PS4 than PS2 (maybe PSX was simpler - PS3 is closer to PS4 and "normal" programming). I've seen numerous examples of how involved CPU was in rendering, especially on PS2, due to special units being exposed as MIPS coprocessors. Efficient rendering meant you had to keep the pipelines happy and that could require significant amount of assembly wrangling. I remember reading about how GTA3 on PS2 was a watershed moment, because it was a proof you could use a somewhat more generic engine - whereas console programming had a heavy amount of assembly focus due to historical reasons.
- giantrobot 5y ago
  I'm sure there's plenty of people on HN with first hand experience developing for the various PlayStation models. I only know what I've read about the experience and technical tear downs. So maybe one will swoop in dropping knowledge bombs and tell me I'm wrong. The original claim was 3D worked on the PSX because of "efficiency" of the code. While I'm not claiming games on the PSX were inefficient, the hardware was designed specifically to do real-time 3D rendering. It's not like the PSX could only do 3D because everyone somehow had super deep mastery of the system. To my understanding the PSX was a big change in console development. While consoles always had some dedicated hardware (blitters, audio DSPs, etc), development was a lot of beam racing. There were no frame buffers, code ran as graphics were being drawn to the screen and during the VBI. The PSX was more like a modern system where you had a CPU and a GPU (albeit fixed function) with an OpenGL-ish drawing model. It was a far more capable 3D machine than PCs of the time that had to do all 3D work on the CPU with no acceleration. Tomb Raider on the PSX was impressive because the console only cost $300. While I'm sure Tomb Raider on the PSX wasn't inefficiently coded, the design of the PSX console contributed as much to its existence as the code. Quake on the PC on the other hand could only exist because of the developers' deep system mastery pushing generic hardware to its limits (before GPUs were prevalent).

quotemstr 5y ago

I'll make a deal with you: I'll agree to turn down speculative execution in hardware if 1) you can solve the VLIW scheduling problem, and 2) you transition the entire computing ecosystem to a JIT model. These are the things we would need to claw back the performance we would lose through disabling speculation. You can look at speculation (I'm handwaving a lot here, bear with me) as the processor hardware dynamically recompiling your program code depending on observed behavior of the program. That's well and good and it gets us a huge performance boost. You can, in principle, do the same thing in pure software. But every single attempt to do has ended in total algorithmic failure. The last serious attempt I'm aware of to "drive" a uOP scheduler explicitly in software was Itanium, and that failed, in part, because compilers couldn't take advantage of the processor's instruction level parallelism. There's nothing in math or mathematics or computer science that forbids a magical compiler of the sort the Itanium people wanted to create. But nobody's made one. Your first task in your project of eliminating speculative execution is to solve this algorithmic problem. But solving problem #1, while necessary, is insufficient. No static ahead-of-time compiler can adjust the compiled code depending on the actual execution history of the program. To really get back to speculative execution par, you have to give your already-magical compiler the ability to recompile code at runtime. That means turning everything into a JIT. Your /bin/ls would actually be LLVM bytecode, not machine code, and some runtime system would be responsible for dynamically generating the machine code and adjusting it depending on execution history. After all, that's what current superscalar CPUs do internally, transparently, all the time. This is problem #2. Honestly, I think the world we'd create by solving both these problems would be a better world. I really don't like how we can't program the processor's speculation engine and uOP scheduler. I'd love to be able to do that. But I don't think we can get there from where we are, so we're going to be stuck with speculation and hardware mitigation forever. Please, prove me wrong.

rmdashrfstar 5y ago
> Honestly, I think the world we'd create by solving both these problems would be a better world. I really don't like how we can't program the processor's speculation engine and uOP scheduler. I'd love to be able to do that. Have you started working on any solutions for 1 and 2?