4 ms·

Amazing to hear from someone so deeply involved in the design process. Thanks for clarifying that some parts of the article are not 100% accurate. Concerning:

by Topfi 3y ago

Amazing to hear from someone so deeply involved in the design process. Thanks for clarifying that some parts of the article are not 100% accurate. Concerning: > [...] really pushing the bounds of what you can do with an in order core and made me realise the out of order/in order boundary is a lot more blurry than you might think. Plenty of out of order stuff happening in the memory system for instance. Are you able to expand on that a bit or point me in a direction on where to get some more information (books, articles, papers)? I'd love to read some more on what can be done to bridge the gap between the two and the specifics of A510 in that regard. Only if you want and are able to (NDA wise) of course. Would you speculate that a future design is going to lean even more heavily/be entirely based around OOE or is such a "hybrid approach" (please excuse my layman terms) still something that can serve certain applications (e.g. space efficiency) well in the future? I understand if you don't want to make a statement one-way or another of course.

gchadwick 3y ago
> Are you able to expand on that a bit or point me in a direction on where to get some more information (books, articles, papers)? No books/articles etc to point to but I can highlight something mentioned in the linked article. > Specifically, the A510 can overlap two cache misses with the following between them: ... 5 loads. The A53 would stall on any memory access past a cache miss. If you're being strictly in order then on a cache miss you'll have to stall waiting for the cache fetch to complete (see the A53 behaviour). The A510 doesn't do that, it can happily service a load (or indeed a store if I'm remembering correctly) that hits in the cache whilst the previous memory access is still on-going. This is out of order behaviour. > Would you speculate that a future design is going to lean even more heavily/be entirely based around OOE or is such a "hybrid approach" (please excuse my layman terms) still something that can serve certain applications (e.g. space efficiency) well in the future? My gut feeling (i.e. I haven't spent time doing a proper analysis here) is that an in-order design that leans on out of order behaviour at a smaller scale (so no big re-order queues massive front end issue windows etc) is the sweet spot for small CPUs. Doing a full out of order design adds a bunch of complexity that costs you power and area and adds to the complexity of verification and risk you'll get a post-silicon bug. I would note that CPU design is a messy business. It's a mistake to try to bin things into strict categories (that often don't have a precise definition anyway!) like RISC v CISC or in order v out of order. When a team is looking at how to push the design to hit some power, performance or area metric they're not caring about the label one might apply, they'll do what they need to hit the targets. So everything ends up as a hybrid design one way or the other.
- ademeure 3y ago
  As a former GPU architect, that's really interesting, thanks! I didn't realise A53's caches were strictly in-order and couldn't service hits ahead of misses, I always assumed this was something even much simpler designs were capable of. I think complexity of verification as an argument against out-of-order is questionable, because if out-of-order resulted in a better core and a competitor did manage to build and properly verify such a core, then they would have a strong competitive advantage. But that might not be true in practice given the area/power cost. As an aside: different GPU vendors also have different limitations when it comes to in-order vs out-of-order caches, and GPUs have the extra complexity that loads are effectively doing "gather", e.g. 32-wide warps doing a load with 32 addresses that may or may not uniquify, so a single "return" to the shader processor may be anything from 1 to 32 (or even 64) cachelines. And GPU gets even more tricky with the texture unit doing trilinear+anisotropic filtering, so a single pixel may require 32x as many inputs, and you may even get into situations where the cache isn't big enough (or doesn't have enough ways) to handle the worst case and you have to revert to in-order for certain modes, or process things at a finer granularity than entire warps! Or just do in-order for everything with huge latency FIFOs and accept the latency cost. Lots of different ways to handle this, also depending on what granularity of returns your shader processor can handle. As you said, both modern CPUs and GPUs can't really be defined using simple labels. Gather makes things a lot harder for load pipelines so I'm not surprised Zen4 seems to still just split it into uOps, but I'm curious exactly how Intel solves handles it in their CPU microarchitecture. Sadly this is the kind of thing that's practically impossible to know as an outsider!
  - artemonster 3y ago
    Can you recommend something to read and learn for an experienced hardware designer (with a bit of graphics pipeline knowledge), if I want to make my own toy GPU? The field seems to be exceptionally interesting and I have no idea how to get in :)

by Topfi 3y ago

gchadwick 3y ago

> Are you able to expand on that a bit or point me in a direction on where to get some more information (books, articles, papers)? No books/articles etc to point to but I can highlight something mentioned in the linked article. > Specifically, the A510 can overlap two cache misses with the following between them: ... 5 loads. The A53 would stall on any memory access past a cache miss. If you're being strictly in order then on a cache miss you'll have to stall waiting for the cache fetch to complete (see the A53 behaviour). The A510 doesn't do that, it can happily service a load (or indeed a store if I'm remembering correctly) that hits in the cache whilst the previous memory access is still on-going. This is out of order behaviour. > Would you speculate that a future design is going to lean even more heavily/be entirely based around OOE or is such a "hybrid approach" (please excuse my layman terms) still something that can serve certain applications (e.g. space efficiency) well in the future? My gut feeling (i.e. I haven't spent time doing a proper analysis here) is that an in-order design that leans on out of order behaviour at a smaller scale (so no big re-order queues massive front end issue windows etc) is the sweet spot for small CPUs. Doing a full out of order design adds a bunch of complexity that costs you power and area and adds to the complexity of verification and risk you'll get a post-silicon bug. I would note that CPU design is a messy business. It's a mistake to try to bin things into strict categories (that often don't have a precise definition anyway!) like RISC v CISC or in order v out of order. When a team is looking at how to push the design to hit some power, performance or area metric they're not caring about the label one might apply, they'll do what they need to hit the targets. So everything ends up as a hybrid design one way or the other.

ademeure 3y ago
As a former GPU architect, that's really interesting, thanks! I didn't realise A53's caches were strictly in-order and couldn't service hits ahead of misses, I always assumed this was something even much simpler designs were capable of. I think complexity of verification as an argument against out-of-order is questionable, because if out-of-order resulted in a better core and a competitor did manage to build and properly verify such a core, then they would have a strong competitive advantage. But that might not be true in practice given the area/power cost. As an aside: different GPU vendors also have different limitations when it comes to in-order vs out-of-order caches, and GPUs have the extra complexity that loads are effectively doing "gather", e.g. 32-wide warps doing a load with 32 addresses that may or may not uniquify, so a single "return" to the shader processor may be anything from 1 to 32 (or even 64) cachelines. And GPU gets even more tricky with the texture unit doing trilinear+anisotropic filtering, so a single pixel may require 32x as many inputs, and you may even get into situations where the cache isn't big enough (or doesn't have enough ways) to handle the worst case and you have to revert to in-order for certain modes, or process things at a finer granularity than entire warps! Or just do in-order for everything with huge latency FIFOs and accept the latency cost. Lots of different ways to handle this, also depending on what granularity of returns your shader processor can handle. As you said, both modern CPUs and GPUs can't really be defined using simple labels. Gather makes things a lot harder for load pipelines so I'm not surprised Zen4 seems to still just split it into uOps, but I'm curious exactly how Intel solves handles it in their CPU microarchitecture. Sadly this is the kind of thing that's practically impossible to know as an outsider!
- artemonster 3y ago
  Can you recommend something to read and learn for an experienced hardware designer (with a bit of graphics pipeline knowledge), if I want to make my own toy GPU? The field seems to be exceptionally interesting and I have no idea how to get in :)