4 ms·

> Are you able to expand on that a bit or point me in a direction on where to get some more information (books, articles, papers)? No books/articles etc to poi

> Are you able to expand on that a bit or point me in a direction on where to get some more information (books, articles, papers)? No books/articles etc to point to but I can highlight something mentioned in the linked article. > Specifically, the A510 can overlap two cache misses with the following between them: ... 5 loads. The A53 would stall on any memory access past a cache miss. If you're being strictly in order then on a cache miss you'll have to stall waiting for the cache fetch to complete (see the A53 behaviour). The A510 doesn't do that, it can happily service a load (or indeed a store if I'm remembering correctly) that hits in the cache whilst the previous memory access is still on-going. This is out of order behaviour. > Would you speculate that a future design is going to lean even more heavily/be entirely based around OOE or is such a "hybrid approach" (please excuse my layman terms) still something that can serve certain applications (e.g. space efficiency) well in the future? My gut feeling (i.e. I haven't spent time doing a proper analysis here) is that an in-order design that leans on out of order behaviour at a smaller scale (so no big re-order queues massive front end issue windows etc) is the sweet spot for small CPUs. Doing a full out of order design adds a bunch of complexity that costs you power and area and adds to the complexity of verification and risk you'll get a post-silicon bug. I would note that CPU design is a messy business. It's a mistake to try to bin things into strict categories (that often don't have a precise definition anyway!) like RISC v CISC or in order v out of order. When a team is looking at how to push the design to hit some power, performance or area metric they're not caring about the label one might apply, they'll do what they need to hit the targets. So everything ends up as a hybrid design one way or the other.

ademeure 3y ago
As a former GPU architect, that's really interesting, thanks! I didn't realise A53's caches were strictly in-order and couldn't service hits ahead of misses, I always assumed this was something even much simpler designs were capable of. I think complexity of verification as an argument against out-of-order is questionable, because if out-of-order resulted in a better core and a competitor did manage to build and properly verify such a core, then they would have a strong competitive advantage. But that might not be true in practice given the area/power cost. As an aside: different GPU vendors also have different limitations when it comes to in-order vs out-of-order caches, and GPUs have the extra complexity that loads are effectively doing "gather", e.g. 32-wide warps doing a load with 32 addresses that may or may not uniquify, so a single "return" to the shader processor may be anything from 1 to 32 (or even 64) cachelines. And GPU gets even more tricky with the texture unit doing trilinear+anisotropic filtering, so a single pixel may require 32x as many inputs, and you may even get into situations where the cache isn't big enough (or doesn't have enough ways) to handle the worst case and you have to revert to in-order for certain modes, or process things at a finer granularity than entire warps! Or just do in-order for everything with huge latency FIFOs and accept the latency cost. Lots of different ways to handle this, also depending on what granularity of returns your shader processor can handle. As you said, both modern CPUs and GPUs can't really be defined using simple labels. Gather makes things a lot harder for load pipelines so I'm not surprised Zen4 seems to still just split it into uOps, but I'm curious exactly how Intel solves handles it in their CPU microarchitecture. Sadly this is the kind of thing that's practically impossible to know as an outsider!
- artemonster 3y ago
  Can you recommend something to read and learn for an experienced hardware designer (with a bit of graphics pipeline knowledge), if I want to make my own toy GPU? The field seems to be exceptionally interesting and I have no idea how to get in :)

ademeure 3y ago

As a former GPU architect, that's really interesting, thanks! I didn't realise A53's caches were strictly in-order and couldn't service hits ahead of misses, I always assumed this was something even much simpler designs were capable of. I think complexity of verification as an argument against out-of-order is questionable, because if out-of-order resulted in a better core and a competitor did manage to build and properly verify such a core, then they would have a strong competitive advantage. But that might not be true in practice given the area/power cost. As an aside: different GPU vendors also have different limitations when it comes to in-order vs out-of-order caches, and GPUs have the extra complexity that loads are effectively doing "gather", e.g. 32-wide warps doing a load with 32 addresses that may or may not uniquify, so a single "return" to the shader processor may be anything from 1 to 32 (or even 64) cachelines. And GPU gets even more tricky with the texture unit doing trilinear+anisotropic filtering, so a single pixel may require 32x as many inputs, and you may even get into situations where the cache isn't big enough (or doesn't have enough ways) to handle the worst case and you have to revert to in-order for certain modes, or process things at a finer granularity than entire warps! Or just do in-order for everything with huge latency FIFOs and accept the latency cost. Lots of different ways to handle this, also depending on what granularity of returns your shader processor can handle. As you said, both modern CPUs and GPUs can't really be defined using simple labels. Gather makes things a lot harder for load pipelines so I'm not surprised Zen4 seems to still just split it into uOps, but I'm curious exactly how Intel solves handles it in their CPU microarchitecture. Sadly this is the kind of thing that's practically impossible to know as an outsider!

artemonster 3y ago
Can you recommend something to read and learn for an experienced hardware designer (with a bit of graphics pipeline knowledge), if I want to make my own toy GPU? The field seems to be exceptionally interesting and I have no idea how to get in :)