4 ms·
They can extract some ILP, but most of that is because it has no limitations but the semantics of the instruction set. It also cheats and can speculate just abo
by DannyBee 9y ago
They can extract some ILP, but most of that is because it has no limitations but the semantics of the instruction set. It also cheats and can speculate just about everything legally, because it can always undo it.
A large amount of the ILP is, AFAIK, from being able to prefetch accurately (IE guess what memory will be accessed) and guess branch directions correctly.
This is not possible to do as well statically.
You actually can do it well dynamically (IE i could make a branch guessing thread and a prefetching thread), but you can't do it fast enough.
In a lot of workloads, most time is spent stalling for memory.
Also note that in VLIW, you have to extract ILP or you are just wasting time (IE are executing NOPs). You end up with a half or 3/4 empty pipeline. So unless you can predict you are going to stall for memory well, you are wasting the pipeline.
On the other hand, modern OOO cores can just extract more ILP when possible. IE when stalling for memory, they can figure out what they could be doing otherwise.
- chrisseaton 9y agoMaybe it's like JITs vs static compilers - we're better off with JIT even if your language is static, because you can adapt to real runtime conditions.
- CyberDildonics 9y agoBut we aren't better off with JIT in software. x64 processors work because they are doing things dynamically in separate hardware. It is part of how they use ever increasing transistor budgets to speed up serial processing.
- bonzini 9y agoReally, at this point the transistor budget is mostly going to bigger caches and more cores. It's really little short of a miracle that Intel still managed to extract double digit percent improvements in serial code for all of Sandy Bridge, Haswell and Skylake.
- deepnotderp 9y agoIt's less of a miracle and more like "larger OoO window, K done"
- bonzini 9y agoNo, that's not so easy. You cannot fill that out-of-order window if you mispredict branches. So the branch prediction has gotten better and better, to an absurd level of accuracy. You also need more execution units, to accommodate more kinds of serial code without stalling the pipeline. In Skylake they also improved the retirement unit, apparently to improve hyperthreading performance. There are also better prefetchers and decoders, while the decoders also become more complicated because this is good old x86's variable-length encoding, with further complications for x86-64, AVX, and so on. TLBs also get more complex (there have been two-level TLBs for a while now), though I guess that is part of the "bigger caches" part.
- deepnotderp 9y agoI agree, I was being a little blunt, but it's not necessarily as complicated as people make it out to be. The best branch predictor is once again the perceptron, and it beats current Intel chips on benchmarks. (I would hazard a, guess that Intel uses something like the tage predictor)
- thesz 9y agoJITs do not and cannot have a complete program view. Whole program compiler can insert JIT-calling code (specializing for common pattern), JIT compiler cannot view whole program. One of the things JITs can't do is to add and/or remove parallelism from the program (i.e., add/remove OpenMP pragmas).
- chrisseaton 9y ago> JITs can't do is to add and/or remove parallelism from the program Of course they can. A trivial practical example is the JVM JIT using vector instructions to add parallelism to a program that didn't have any before. But a JIT compiler could also profile code and add thread-level parallelism if it wanted.
- thesz 9y agoFor thread-level parallelism to be added you need to perform something akin to Fourier-Motzkin resolution algorithm or Omega test to find out what dependencies program has and has not. FM is NP-hard. Omega test is imprecise. And in general you need more complicated checks like aliasing checks. Which are equal to stopping problem, most of the time. Usually you can't spend that amount of time on EACH INVOCATION of a program.
- chrisseaton 9y agoThe techniques you are talking about are for proving that there is parallelism. The beauty of a JIT is that you can speculate that there is parallelism, go ahead, and then sort out the mess later and recompile to sequential if there wasn't. https://en.wikipedia.org/wiki/Speculative_multithreading https://en.wikipedia.org/wiki/Speculative_multithreading You can use hardware functionality such as TSX to support this.
- thesz 9y agoYou can't speculate about parallelism when dealing with shared resource. It is unbelievable you even suggested it. TSX won't save you.
- poopsicle 9y agoIt sounds like you just proved the point.
- lobster_johnson 9y agoI always wondered how CPUs could do all of this so fast. Doesn't all the branch prediction etc. have to be done by a kind of program that is "parsing" instructions on the fly and storing state about its execution? How is that implemented, and how is it faster than doing the equivalent in software?
- deepnotderp 9y agoBranch prediction under latency constraints is a difficult task! I believe perceptrons are the SOTA right now (have recently beat out TAGE predictors to take the crown once again) but are very energy intensive, the alternative is (which is currently used I think) are two level look up tables.