3 ms·
I wonder why do we have to rely on CPU to parallelize code. Surely compiler could do better job and CPU should just offer transistors without any smart logic. I
by vbezhenar 8y ago
I wonder why do we have to rely on CPU to parallelize code. Surely compiler could do better job and CPU should just offer transistors without any smart logic. It's not backwards-compatible and I'm aware about Itanium fiasco, but I'm not compelled that it's a wrong way.
- tylerhou 8y agoA quote I once heard from a friend: "Don't turn your normal problem into a distributed systems problem."
- mokus 8y agoArguably, the reason for Spectre, etc., is that people have failed to realize that at the scale of modern intel CPUs, these problems already are distributed systems problems.
- jcranmer 8y agoAs someone who works in compilers: no, it's not possible to make the compiler do a better job. There's a reason why VLIW architectures keep getting proposed and keep dying. The instruction-level parallelism that a CPU can extract is primarily a dynamic kind of parallelism. You can, say, have a branch that's true 1000 times, then false 1000 times, then true 1000 times, then false 1000 times, etc.--as a compiler, telling the hardware to predict true or false is going to guarantee a 50% hit rate, but the stupid simple dynamic branch predictor will get 99% on that branch.
- kinghajj 8y agoWhat are your thoughts on the Mill CPU team's claims re: their ISA allowing compilers feasibly to schedule operations statically?
- Taniwha 8y agoyou can do this to some degree on most CPU's - moving loads away from their results being used - compilers, esp on RISC machines with lots of registers, do this today VLIW machines allow you to provide hints about instruction level parallelism without all the superscalar on-the-fly analysis of instruction level data dependencies (so the hardware can do all that rescheduling on the fly). I think that if interlock-free software scheduled CPUs allowed us to reach 20GHz clock speeds where complex superscalar machines were stuck at 3GHz we'd all be jumping ship - but they're not
- AstralStorm 8y agoEssentially the CPU is running a JIT branch optimization and speculative loads which cannot ever be matched by static optimization. The interesting part is that any JIT compiler that is so dynamic in performance will have these problems. The solution might be to actually prevent some kind of security important code from being optimized in this way. Say, forcing full cache sync in-order execution for parts of code with no resource sharing between cores.
- Taniwha 8y agohmm ... I wonder if there are JiT compilers with smallish caches that are susceptible to these attacks (but over much larger time scales)
- blattimwind 8y ago> I think that if interlock-free software scheduled CPUs allowed us to reach 20GHz clock speeds where complex superscalar machines were stuck at 3GHz we'd all be jumping ship - but they're not Additionally, realize that clock speeds are not a relevant performance metric. Maybe, perhaps, it's possible for a software scheduled CPU to run at 2*n GHz, but that's not interesting, if it's slower in real-world workloads than todays tech at n GHz. And for breadth-deployment in data centers we're mostly looking at throughput per Watt. CPUs with high clock speeds don't do well in that area because semiconductor physics.
- 8y ago
- Symmetry 8y agoThey execute everything in order on a fairly short pipeline but they engage in speculative execution just like everyone else. Here's their talk on their branch predictor. https://millcomputing.com/docs/prediction/ https://millcomputing.com/docs/prediction/ There's a lot less mischief you can get up to when you only execute a couple of instructions beyond a mispredicted branch rather than a hundred but mischief can't be theoretically ruled out entirely.
- aseipp 8y agoWell, there are middle grounds, such as EDGE architectures, that get you somewhere in the middle of a VLIW and CISC machine, where everything is not so rigid. The trick isn't that the compiler must do better in scheduling things perfectly against unknown, dynamic information -- but must emit the information it already knows, and they currently throw away. This is how TRIPS worked, where the compiler placed instructions statically like a VLIW machine (it was a grid-like architecture, so this is obviously important), but the CPU issued instructions dynamically at the basic block level, like a OoO CISC machine. The secret is you encode the dataflow dependencies inside basic blocks, more or less -- so the CPU does not have to rediscover them. I worked on compilers too, and I think there is definitely still work to be explored here. Really, complicated OoO speculative processors are simply recovering a lot of the information the compiler already calculated! You take this code that is in no way suited for a CPU, and you do a ton of dataflow and high-level analysis on it (that you can only know from the source). You build this graph and optimized based on these facts. And then, you throw away the dataflow graph when you lower things. All this, after painfully calculating it on the assumed basis of "Yes, the CPU will like this code" -- only for the CPU to perform that whole process over again, say "Yes, I do, in fact, like this code" -- so it can execute efficiently without stalls anyway. There's clearly a mismatch here. I mean, don't get me wrong -- this all seems like a perfectly fine and acceptable engineering tradeoff, but just disappointing from a computer science perspective, to me, at least, that this is not unified :) Of course, VLIW vs CISC etc is one of the classic debates... Interestingly, Aaron Smith from Microsoft Research (and one of the original members of the TRIPS project) actually demonstrated and talked about their work on newer EDGE processor designs at MSR, and even demonstrated Windows running on an EDGE processor, this past week or so at ISCA2018 (complete with a working Visual Studio target, using an LLVM-based toolchain!) Their design is quite different from TRIPS it seems. I'm hoping the talks/work becomes public eventually, but that might just be a dream.
- jcranmer 8y agoI should clarify that the idea that is wrong is that just exposing the hardware bits to the compiler and expecting the compiler to do better. There is certainly more scope for the compiler and the hardware to work together to make the results better, but exposing current hardware mechanisms is not the means to do to that. Of course, try getting architecture and compiler people actually talking to each other. :-)
- Symmetry 8y agoVLIWs can make use of the same branch predictors that you see in out of order cores like an Intel Core or in order cores like an ARM A53. The big problem that VLIW faces is starting loads early so that they've finished by the time you need their contents. Itanium tried to do that but their mechanism really broke down in the face of possible memory exceptions and it didn't speed things up much. Mill has a cleverer way to speculate with loads which might work better but it's still entirely unable to speculatively load values across boundaries between codebases, like system calls. But of course speculatively loading across system calls is what's causing problems here. So it might be that the best thing is to accept the 80% performance solution in the same of security through keeping the hardware simple enough to understand.
- Tuna-Fish 8y agoThe compiler cannot compete with the CPU because the CPU has strictly more information. Any memory access can take between 4 and thousands of cycles of time to execute, and the compiler won't know how long any of them take. The CPU can reorder based on exact knowledge.
- Symmetry 8y agoUsing profile guided optimization can help a lot with this in practice, though it isn't perfect.