4 ms·

You have your facts almost completely backwards. Compilers don't know best and any company who has banked on that for general purpose CPUs has been burned hard

You have your facts almost completely backwards. Compilers don't know best and any company who has banked on that for general purpose CPUs has been burned hard. Your explanation of what a deferred load is, is actually more of a description of prefetching. The compiler also does not necessarily know instruction latencies since they change from one CPU to the next. Out of order execution already does the job of deferred loads. Loads can be executed as soon as they are seen and other instructions can be run later when their needed memory has made it to the CPU. This is why Haswell already had a 192 instruction out of order buffer. OoO execution also schedules instructions to be run on the multiple execution units and ends up doing what compilers were supposed to do with VLIW CPUs. > "Prefetching is an optimization of caching, it still doesn't beat the speed of signal in a silicon CPU, nor does it solve the scalability issues mentioned in the article." None of this is true. Prefetching looks at access patterns and pulls down sections of memory before they are accessed. Caching is about saving what has already been accessed. I'm not sure what you mean by 'beating the speed of signal' but if you are talking about latency, that is exactly what it deals with. By the time memory is needed it is already over to the CPU. The article talks about issues that are due to memory latency (which much of modern CPUs features deal with on way or another) and prefetching directly confronts this. Instruction access that happens linearly can be prefetched.

CalChris 7y ago
There's SW prefetch instructions and HW prefetching engines. SW prefetch instructions have largely been a bust. Linus is famous for ranting about them. https://yarchive.net/comp/linux/software_prefetching.html https://yarchive.net/comp/linux/software_prefetching.html HW looks at access patterns (as you say) and does at least as good a job.
- BubRoss 7y ago
  Yep, I've tried to use prefetch instrinsics multiple times and I've never been able to beat the CPU and speed things up.
zaarn 7y ago
>The compiler also does not necessarily know instruction latencies since they change from one CPU to the next. The instruction latencies are actually all documented and modern compiler do take care when generating assembly code to optimize the output a bit to not stall the pipeline too often. I don't see how compilers, which have greatly improved since Itanium, shouldn't be able to tell when a load is best issued and be able to keep the pipelines full. >None of this is true. Prefetching looks at access patterns and pulls down sections of memory before they are accessed. That pretty much sounds like an optimization to caching for me. Granted, it's not precisely a caching algorithm but it's more of a cache predictor, but cache in the end. Compiler know best how to order instructions to ensure that loads happen when best needed and sufficiently beforehand that the data arrives in time without having to stall any part of the pipeline. The CPU solves it at runtime, why can't the compiler do it at compiletime when all the same information is available?

CalChris 7y ago

There's SW prefetch instructions and HW prefetching engines. SW prefetch instructions have largely been a bust. Linus is famous for ranting about them. https://yarchive.net/comp/linux/software_prefetching.html https://yarchive.net/comp/linux/software_prefetching.html HW looks at access patterns (as you say) and does at least as good a job.

BubRoss 7y ago
Yep, I've tried to use prefetch instrinsics multiple times and I've never been able to beat the CPU and speed things up.

zaarn 7y ago

>The compiler also does not necessarily know instruction latencies since they change from one CPU to the next. The instruction latencies are actually all documented and modern compiler do take care when generating assembly code to optimize the output a bit to not stall the pipeline too often. I don't see how compilers, which have greatly improved since Itanium, shouldn't be able to tell when a load is best issued and be able to keep the pipelines full. >None of this is true. Prefetching looks at access patterns and pulls down sections of memory before they are accessed. That pretty much sounds like an optimization to caching for me. Granted, it's not precisely a caching algorithm but it's more of a cache predictor, but cache in the end. Compiler know best how to order instructions to ensure that loads happen when best needed and sufficiently beforehand that the data arrives in time without having to stall any part of the pipeline. The CPU solves it at runtime, why can't the compiler do it at compiletime when all the same information is available?