5 ms·

That sounds a bit dismissive. Deferred loads do have a massive advantage; your compiler knows best when loads are needed, so it can, for example, run the load

by zaarn 7y ago

That sounds a bit dismissive. Deferred loads do have a massive advantage; your compiler knows best when loads are needed, so it can, for example, run the load for the next array value while still processing the current one, allowing the CPU to not stall at all at highest efficiency (because the compiler can know instruction latencies and counts out when to start the load). Prefetching is an optimization of caching, it still doesn't beat the speed of signal in a silicon CPU, nor does it solve the scalability issues mentioned in the article.

heavenlyblue 7y ago
The bandwidth of memory bus is still a limiting factor.
- zaarn 7y ago
  I don't think memory bandwidth is the limiting factor for a modern CPU. If instruction fetching was not only consuming most bandwidth but also dominated it, then there would be little benefit to SIMD if you could just do multiple instructions instead as the bandwidth is taken up anyways (notably in multicore systems). That plus to my knowledge, there is no single core performance difference at same clock between a 4-core and 12-core CPU in AMD CPUs, which would be the case if instruction fetching is maxing out memory bandwidth.
- jdsully 7y ago
  I find latency to be the much bigger problem. For the types of workloads where bandwidth is limiting you tend to find they run much better on GPUs.
gpderetta 7y ago
> compiler knows best when loads are needed History has shown repitedly that in fact compilers do not know best outside some very specific use cases. Whether a load need to be issued early and how early is a dynamic property of a program and it is hard to compute statically. Prefetching and deferred loads have existed for a long time but they didn't help, for eg. Itanium.
- zaarn 7y ago
  How is loading a value from memory when an appropriate opcode comes in a dynamic property? Mill seems to be doing fine with their compiler and since Itanium compilers have gotten a lot smarter.
  - gpderetta 7y ago
    Whether a load is fulfilled from L1, L2, L3 or main memory is a dynamic property of the program and very hard to predict statically. Aliasing is also a dynamic property, but read ahead hardware should be able to handle it (but failure might stall the whole pipeline). Saying that Mill doing is fine might be tiny bit optimistic seening that the whole thing is still vaporware. Do they have a working compiler now?
    - zaarn 7y ago
      Atleast judging from the latest posts on their forums, they have a working C compiler and are working on a C++ compiler to get a basic kernel for test harnesses running. I don't see why it matters to a deferred load being issued if the data is in L1, L2 or L3. The compiler emits a load, the caching hardware handles the rest, once the load arrives the pipeline unstalls. The compiler can atleast assume a L1 cache latency and then put some cycles between the load and execution. The earlier you can issue the load the better, if too early you might ruin it if you get preempted. Either way, it should not be hard to solve in VLIW.
BubRoss 7y ago
You have your facts almost completely backwards. Compilers don't know best and any company who has banked on that for general purpose CPUs has been burned hard. Your explanation of what a deferred load is, is actually more of a description of prefetching. The compiler also does not necessarily know instruction latencies since they change from one CPU to the next. Out of order execution already does the job of deferred loads. Loads can be executed as soon as they are seen and other instructions can be run later when their needed memory has made it to the CPU. This is why Haswell already had a 192 instruction out of order buffer. OoO execution also schedules instructions to be run on the multiple execution units and ends up doing what compilers were supposed to do with VLIW CPUs. > "Prefetching is an optimization of caching, it still doesn't beat the speed of signal in a silicon CPU, nor does it solve the scalability issues mentioned in the article." None of this is true. Prefetching looks at access patterns and pulls down sections of memory before they are accessed. Caching is about saving what has already been accessed. I'm not sure what you mean by 'beating the speed of signal' but if you are talking about latency, that is exactly what it deals with. By the time memory is needed it is already over to the CPU. The article talks about issues that are due to memory latency (which much of modern CPUs features deal with on way or another) and prefetching directly confronts this. Instruction access that happens linearly can be prefetched.
- CalChris 7y ago
  There's SW prefetch instructions and HW prefetching engines. SW prefetch instructions have largely been a bust. Linus is famous for ranting about them. https://yarchive.net/comp/linux/software_prefetching.html https://yarchive.net/comp/linux/software_prefetching.html HW looks at access patterns (as you say) and does at least as good a job.
  - BubRoss 7y ago
    Yep, I've tried to use prefetch instrinsics multiple times and I've never been able to beat the CPU and speed things up.
- zaarn 7y ago
  >The compiler also does not necessarily know instruction latencies since they change from one CPU to the next. The instruction latencies are actually all documented and modern compiler do take care when generating assembly code to optimize the output a bit to not stall the pipeline too often. I don't see how compilers, which have greatly improved since Itanium, shouldn't be able to tell when a load is best issued and be able to keep the pipelines full. >None of this is true. Prefetching looks at access patterns and pulls down sections of memory before they are accessed. That pretty much sounds like an optimization to caching for me. Granted, it's not precisely a caching algorithm but it's more of a cache predictor, but cache in the end. Compiler know best how to order instructions to ensure that loads happen when best needed and sufficiently beforehand that the data arrives in time without having to stall any part of the pipeline. The CPU solves it at runtime, why can't the compiler do it at compiletime when all the same information is available?

by zaarn 7y ago

heavenlyblue 7y ago

The bandwidth of memory bus is still a limiting factor.

zaarn 7y ago
I don't think memory bandwidth is the limiting factor for a modern CPU. If instruction fetching was not only consuming most bandwidth but also dominated it, then there would be little benefit to SIMD if you could just do multiple instructions instead as the bandwidth is taken up anyways (notably in multicore systems). That plus to my knowledge, there is no single core performance difference at same clock between a 4-core and 12-core CPU in AMD CPUs, which would be the case if instruction fetching is maxing out memory bandwidth.
jdsully 7y ago
I find latency to be the much bigger problem. For the types of workloads where bandwidth is limiting you tend to find they run much better on GPUs.

gpderetta 7y ago

> compiler knows best when loads are needed History has shown repitedly that in fact compilers do not know best outside some very specific use cases. Whether a load need to be issued early and how early is a dynamic property of a program and it is hard to compute statically. Prefetching and deferred loads have existed for a long time but they didn't help, for eg. Itanium.

zaarn 7y ago
How is loading a value from memory when an appropriate opcode comes in a dynamic property? Mill seems to be doing fine with their compiler and since Itanium compilers have gotten a lot smarter.
- gpderetta 7y ago
  Whether a load is fulfilled from L1, L2, L3 or main memory is a dynamic property of the program and very hard to predict statically. Aliasing is also a dynamic property, but read ahead hardware should be able to handle it (but failure might stall the whole pipeline). Saying that Mill doing is fine might be tiny bit optimistic seening that the whole thing is still vaporware. Do they have a working compiler now?
  - zaarn 7y ago
    Atleast judging from the latest posts on their forums, they have a working C compiler and are working on a C++ compiler to get a basic kernel for test harnesses running. I don't see why it matters to a deferred load being issued if the data is in L1, L2 or L3. The compiler emits a load, the caching hardware handles the rest, once the load arrives the pipeline unstalls. The compiler can atleast assume a L1 cache latency and then put some cycles between the load and execution. The earlier you can issue the load the better, if too early you might ruin it if you get preempted. Either way, it should not be hard to solve in VLIW.

BubRoss 7y ago

You have your facts almost completely backwards. Compilers don't know best and any company who has banked on that for general purpose CPUs has been burned hard. Your explanation of what a deferred load is, is actually more of a description of prefetching. The compiler also does not necessarily know instruction latencies since they change from one CPU to the next. Out of order execution already does the job of deferred loads. Loads can be executed as soon as they are seen and other instructions can be run later when their needed memory has made it to the CPU. This is why Haswell already had a 192 instruction out of order buffer. OoO execution also schedules instructions to be run on the multiple execution units and ends up doing what compilers were supposed to do with VLIW CPUs. > "Prefetching is an optimization of caching, it still doesn't beat the speed of signal in a silicon CPU, nor does it solve the scalability issues mentioned in the article." None of this is true. Prefetching looks at access patterns and pulls down sections of memory before they are accessed. Caching is about saving what has already been accessed. I'm not sure what you mean by 'beating the speed of signal' but if you are talking about latency, that is exactly what it deals with. By the time memory is needed it is already over to the CPU. The article talks about issues that are due to memory latency (which much of modern CPUs features deal with on way or another) and prefetching directly confronts this. Instruction access that happens linearly can be prefetched.

CalChris 7y ago
There's SW prefetch instructions and HW prefetching engines. SW prefetch instructions have largely been a bust. Linus is famous for ranting about them. https://yarchive.net/comp/linux/software_prefetching.html https://yarchive.net/comp/linux/software_prefetching.html HW looks at access patterns (as you say) and does at least as good a job.
- BubRoss 7y ago
  Yep, I've tried to use prefetch instrinsics multiple times and I've never been able to beat the CPU and speed things up.
zaarn 7y ago
>The compiler also does not necessarily know instruction latencies since they change from one CPU to the next. The instruction latencies are actually all documented and modern compiler do take care when generating assembly code to optimize the output a bit to not stall the pipeline too often. I don't see how compilers, which have greatly improved since Itanium, shouldn't be able to tell when a load is best issued and be able to keep the pipelines full. >None of this is true. Prefetching looks at access patterns and pulls down sections of memory before they are accessed. That pretty much sounds like an optimization to caching for me. Granted, it's not precisely a caching algorithm but it's more of a cache predictor, but cache in the end. Compiler know best how to order instructions to ensure that loads happen when best needed and sufficiently beforehand that the data arrives in time without having to stall any part of the pipeline. The CPU solves it at runtime, why can't the compiler do it at compiletime when all the same information is available?