3 ms·

Spectre makes me wish we'd give something like IBM's project DAISY / Transmeta's Crusoe another chance. If we have one mode with a simple and high-density inst

by KMag 6y ago

Spectre makes me wish we'd give something like IBM's project DAISY / Transmeta's Crusoe another chance. If we have one mode with a simple and high-density instruction set similar to Hitachi's SH4 / ARM's Thumb2, along with a VLIW mode (similar to switching between Thumb2 and regular ARM), and some hardware support for tracing and dynamic recompilation (including reservoir sampling of instructions causing pipeline stalls), we might get decent performance and code density, while being able to move all of the speculation into the dynamic recompilation layer. Having the processor natively support the dense non-VLIW instruction set means you don't pay much in the way of startup latency, and once you're warmed up, your hot spots have all of their cross-DLL calls inlined / no longer indirect, and your virtual function calls devirtualized / speculative inlined. Spots where the dynamic recompiler was wrong about pipeline stalls eventually use dynamic information to recompile and shuffle instructions around to avoid pipeline stalls. You don't pay the transistor and power budget for out-of-order execution or speculative execution hardware. Hardware speculative execution gets replaced with predicated instructions and shadow register save/restores. I think IBM's DAISY was much closer to being on the right track vs. Intel's EPIC / Itanium. If your system is built for dynamic recompilation and re-optimization of native code on the fly, your compiler doesn't have to be as good at statically predicting execution paths and pipeline stalls. Transmeta's main problem was that they were emulating all x86 instructions before they warmed up, and after warm-up they were still emulating all x86 instructions outside their hot code path. With a simpler instruction set like SH4 / Thumb2, they could hopefully have hardware support for the code outside of the hot spots / pre-warmup. Given the amount of time spent running JavaScript, hopefully one would also look at the bytecodes for V8, JavaScriptCore, and SpiderMonkey for inspiration as far as making the non-VLIW instruction set an efficient and compact JavaScript JIT target.

titzer 6y ago
> If we have one mode with a simple and high-density instruction set similar to Hitachi's SH4 / ARM's Thumb2, along with a VLIW mode (similar to switching between Thumb2 and regular ARM), and some hardware support for tracing and dynamic recompilation (including reservoir sampling of instructions causing pipeline stalls), we might get decent performance and code density, while being able to move all of the speculation into the dynamic recompilation layer. Trading hardware complexity for software complexity is not the right strategy to combat sidechannels. They just reappear up the stack. Any speculative optimization that depends on program values can potentially leak information through timing.
- nine_k 6y ago
  With speculation moved to a (JIT) compiler, this becomes way harder to trigger. You can no longer deterministically trigger a speculative fetch and detect whether it was slow or served from cache. Whether the speculative fetch ever occurs is deteremined by the JIT, and once the JIT realized it should not even occur, it will never occur again. Quite likely you won't be able to collect enough bits even if speculative execution is supported by the hardware. It it's not, you will just not have anything to time, AFAICT.
  - titzer 6y ago
    It's true that it is much, much harder to trigger, and less predictable, but the information leak is still there. Both the bitrate and the signal-to-noise ratio are far worse. It's worth pointing out that JITs can and do get caught in deopt loops, and it doesn't even necessarily need to be a deopt loop in a JIT that is the information leak. Something as simple as interned strings can leak information about what strings a program is using. Also with hashtables; if you have control over some of the keys that go into a hashtable and some knowledge of its implementation, timing information can reveal information about hash collisions, and thus other keys in the table. Fundamentally, side channels are unwanted information flows in a system, and the more complicated the system, the more potential there is for side channels. Moving complexity around might make a difference on the reliability and bandwidth, but it doesn't eliminate them. If side channels are a serious concern, the best defense is simplicity, not massive rearchitecting to a new and different kind of complexity. (even speaking as a person who spent decades working on JITs--wrong hammer here)
    - KMag 6y ago
      My understanding is that Spectre/Meltdown attacks are all the result of either conditional memory operations conditioned upon speculatively read data, or else speculative data operations using addresses calculated from data that was speculatively loaded. Is that correct? Doesn't moving speculation from hardware control to software control allow you to perform better dataflow analysis than you can reasonably do in hardware, which allows you to perform more provably safe speculations (or less often pay the I/O overhead of converting a speculative conditional load into an unconditional load and a speculative conditional register-to-register move) than you get if the speculation decisions need to be performed in hardware?
ansible 6y ago
> ...while being able to move all of the speculation into the dynamic recompilation layer. Or move it all into static analysis. You might be interested in the Mill CPU architecture: https://millcomputing.com/docs/ https://millcomputing.com/docs/ Very Very Very Long Instruction Word system, without a conventional register file. All of the speculation that is done in hardware on a conventional CPU is pushed back to the compiler. More about Spectre, Meltdown and the Mill here: https://millcomputing.com/blog/wp-content/uploads/2018/01/Spectre.03.pdf https://millcomputing.com/blog/wp-content/uploads/2018/01/Sp...
- KMag 6y ago
  Yea, that sort of dataflow analysis (at static compile or just-in-time compile time) is what I'm getting at for avoiding Spectre/Meltdown, but with the added ability of being able to use profiling information to re-generate the code paths where you're getting pipeline stalls due to flaws in the assumptions used in static instruction scheduling.

by KMag 6y ago

titzer 6y ago

> If we have one mode with a simple and high-density instruction set similar to Hitachi's SH4 / ARM's Thumb2, along with a VLIW mode (similar to switching between Thumb2 and regular ARM), and some hardware support for tracing and dynamic recompilation (including reservoir sampling of instructions causing pipeline stalls), we might get decent performance and code density, while being able to move all of the speculation into the dynamic recompilation layer. Trading hardware complexity for software complexity is not the right strategy to combat sidechannels. They just reappear up the stack. Any speculative optimization that depends on program values can potentially leak information through timing.

nine_k 6y ago
With speculation moved to a (JIT) compiler, this becomes way harder to trigger. You can no longer deterministically trigger a speculative fetch and detect whether it was slow or served from cache. Whether the speculative fetch ever occurs is deteremined by the JIT, and once the JIT realized it should not even occur, it will never occur again. Quite likely you won't be able to collect enough bits even if speculative execution is supported by the hardware. It it's not, you will just not have anything to time, AFAICT.
- titzer 6y ago
  It's true that it is much, much harder to trigger, and less predictable, but the information leak is still there. Both the bitrate and the signal-to-noise ratio are far worse. It's worth pointing out that JITs can and do get caught in deopt loops, and it doesn't even necessarily need to be a deopt loop in a JIT that is the information leak. Something as simple as interned strings can leak information about what strings a program is using. Also with hashtables; if you have control over some of the keys that go into a hashtable and some knowledge of its implementation, timing information can reveal information about hash collisions, and thus other keys in the table. Fundamentally, side channels are unwanted information flows in a system, and the more complicated the system, the more potential there is for side channels. Moving complexity around might make a difference on the reliability and bandwidth, but it doesn't eliminate them. If side channels are a serious concern, the best defense is simplicity, not massive rearchitecting to a new and different kind of complexity. (even speaking as a person who spent decades working on JITs--wrong hammer here)
  - KMag 6y ago
    My understanding is that Spectre/Meltdown attacks are all the result of either conditional memory operations conditioned upon speculatively read data, or else speculative data operations using addresses calculated from data that was speculatively loaded. Is that correct? Doesn't moving speculation from hardware control to software control allow you to perform better dataflow analysis than you can reasonably do in hardware, which allows you to perform more provably safe speculations (or less often pay the I/O overhead of converting a speculative conditional load into an unconditional load and a speculative conditional register-to-register move) than you get if the speculation decisions need to be performed in hardware?

ansible 6y ago

> ...while being able to move all of the speculation into the dynamic recompilation layer. Or move it all into static analysis. You might be interested in the Mill CPU architecture: https://millcomputing.com/docs/ https://millcomputing.com/docs/ Very Very Very Long Instruction Word system, without a conventional register file. All of the speculation that is done in hardware on a conventional CPU is pushed back to the compiler. More about Spectre, Meltdown and the Mill here: https://millcomputing.com/blog/wp-content/uploads/2018/01/Spectre.03.pdf https://millcomputing.com/blog/wp-content/uploads/2018/01/Sp...

KMag 6y ago
Yea, that sort of dataflow analysis (at static compile or just-in-time compile time) is what I'm getting at for avoiding Spectre/Meltdown, but with the added ability of being able to use profiling information to re-generate the code paths where you're getting pipeline stalls due to flaws in the assumptions used in static instruction scheduling.