4 ms·

It might be interesting to know if these lanes are per logical core, physical core or CPU. If it's per core, it might be hard to find real-life code using more

It might be interesting to know if these lanes are per logical core, physical core or CPU. If it's per core, it might be hard to find real-life code using more than 10 lanes outside of micro-benchmarks.

mooman219 8y ago
I would bet that Intel has decent statistics on the average number of lanes used, and opted for 10 based on that. Since die space is a premium, they either used the extra space where it's needed more, or shrunk the die by some small amount.
nicoburns 8y ago
I'm not an expert (so I could be completely wrong), but I could imagine in-memory data stores being bound by this kind of limit...
- dragontamer 8y ago
  AMD processors have 44-stores in flight, but only 22-loads in flight. I presume Intel processors have a similar difference between their "store buffers" and their "load buffers".
  - nkurz 8y ago
    I'm not sure if we are using the same terminology, but I think Intel has the opposite arrangement, with more "load buffers" than "store buffers". In Table 2-7. L1 Data Cache Components from https://www.intel.com/content/dam/doc/manual/64-ia-32-architectures-optimization-manual.pdf https://www.intel.com/content/dam/doc/manual/64-ia-32-archit..., it says: Sandy Bridge Load buffers 64 entries Store buffers 36 entries Line fill buffers (LFB) 10 entries I think Haswell (and later?) are 72/42/10 per physical(?) core, also with more load than store. How does this match up with what you know about AMD? And do you have a clear number of LFB's (or whatever they call the buffers which are allocated only for L1D misses) for AMD? Edit: I now see the link you gave in your other comment. Hmm. I think numbers you gave do correspond to the inverted ratio numbers for Intel, and I couldn't find any claimed number corresponding to LFB's. We do have a couple AMD machine we test on, and I think we did, but I don't recall the exact numbers, other than a general recollection that they were about about the same as the Intel. In his RWT post, Travis claims Ryzen as having 8-10.
    - dragontamer 8y ago
      > In his RWT post, Travis claims Ryzen as having 8-10. I wonder if they're statically partitioned: so that you only get ~11 per thread on Ryzen when SMT is activated. The AMD Optimization manual is cler that the LS unit handles cache misses. Intel's manual is clear that LFB handles cache misses. Load / store buffers seem to be a more general set of micro-op reordering in general. -------- It appears I've been mistaken btw. Let me post the full text of the section: > The LS unit includes a 44-entry load queue (LDQ). The LDQ receives load operations at dispatch. Loads leave the LDQ when the load has completed and delivered data to the integer unit or the floating-point unit. > The LS unit utilizes a 44-entry store queue which holds stores from dispatch until the store data can be written to the data cache ... > The LS unit can track up to 22 outstanding in-flight cache misses. So that's 44 load queue, 44 store queue, and 22-cache misses for AMD Ryzen. Presumably, this is split somehow in hyperthreads (which would lead to the ~10 number you suggested)
      - BeeOnRope 8y ago
        Load and store queues are different than the "miss buffers" we are talking about here. Every load and store will need one of these entries from allocation until retirement, so there are usually a lot of them, something like 20% to 40% the size of the ROB (re-order buffer) each, since many code sequences have many loads and stores. The miss buffers are different in that they only get involved when there is an L1 miss. They are only used from the moment the miss is detected until the miss completes. Also, load and store buffers are used 1:1 with load and store instructions (including instructions that have a load, store or both combined with an ALU operation, such as load-op or RMW on x86). Miss buffers are 1:1 with outstanding misses on the granularity of cache lines. So if you load 8 consecutive 4-byte values, which all fall in the same cache line, which misses in L1, you'll have 8 load buffer entries used, but only 1 fill buffer used since the buffer coalesces all misses to the same line.
BeeOnRope 8y ago
The simple answer is that these lanes (buffers, I prefer to call them) are per logical core. The more complicated answer is that there are a variety of buffers involved in the "path to memory", starting with the core-private buffers between the core and L1 and L2, and moving to socket-shared buffers at the L3, memory controllers and DRAM. In this particular single-threaded test, it is the 10 core-private "line fill buffers" between the L1 and L2 which are the bottleneck, which is where you get the simple answer from. If you ran this test on enough cores at the same time, I would entirely expect the bottlneck to change to one of the shared buffers.

mooman219 8y ago

I would bet that Intel has decent statistics on the average number of lanes used, and opted for 10 based on that. Since die space is a premium, they either used the extra space where it's needed more, or shrunk the die by some small amount.

nicoburns 8y ago

I'm not an expert (so I could be completely wrong), but I could imagine in-memory data stores being bound by this kind of limit...

dragontamer 8y ago
AMD processors have 44-stores in flight, but only 22-loads in flight. I presume Intel processors have a similar difference between their "store buffers" and their "load buffers".
- nkurz 8y ago
  I'm not sure if we are using the same terminology, but I think Intel has the opposite arrangement, with more "load buffers" than "store buffers". In Table 2-7. L1 Data Cache Components from https://www.intel.com/content/dam/doc/manual/64-ia-32-architectures-optimization-manual.pdf https://www.intel.com/content/dam/doc/manual/64-ia-32-archit..., it says: Sandy Bridge Load buffers 64 entries Store buffers 36 entries Line fill buffers (LFB) 10 entries I think Haswell (and later?) are 72/42/10 per physical(?) core, also with more load than store. How does this match up with what you know about AMD? And do you have a clear number of LFB's (or whatever they call the buffers which are allocated only for L1D misses) for AMD? Edit: I now see the link you gave in your other comment. Hmm. I think numbers you gave do correspond to the inverted ratio numbers for Intel, and I couldn't find any claimed number corresponding to LFB's. We do have a couple AMD machine we test on, and I think we did, but I don't recall the exact numbers, other than a general recollection that they were about about the same as the Intel. In his RWT post, Travis claims Ryzen as having 8-10.
  - dragontamer 8y ago
    > In his RWT post, Travis claims Ryzen as having 8-10. I wonder if they're statically partitioned: so that you only get ~11 per thread on Ryzen when SMT is activated. The AMD Optimization manual is cler that the LS unit handles cache misses. Intel's manual is clear that LFB handles cache misses. Load / store buffers seem to be a more general set of micro-op reordering in general. -------- It appears I've been mistaken btw. Let me post the full text of the section: > The LS unit includes a 44-entry load queue (LDQ). The LDQ receives load operations at dispatch. Loads leave the LDQ when the load has completed and delivered data to the integer unit or the floating-point unit. > The LS unit utilizes a 44-entry store queue which holds stores from dispatch until the store data can be written to the data cache ... > The LS unit can track up to 22 outstanding in-flight cache misses. So that's 44 load queue, 44 store queue, and 22-cache misses for AMD Ryzen. Presumably, this is split somehow in hyperthreads (which would lead to the ~10 number you suggested)
    - BeeOnRope 8y ago
      Load and store queues are different than the "miss buffers" we are talking about here. Every load and store will need one of these entries from allocation until retirement, so there are usually a lot of them, something like 20% to 40% the size of the ROB (re-order buffer) each, since many code sequences have many loads and stores. The miss buffers are different in that they only get involved when there is an L1 miss. They are only used from the moment the miss is detected until the miss completes. Also, load and store buffers are used 1:1 with load and store instructions (including instructions that have a load, store or both combined with an ALU operation, such as load-op or RMW on x86). Miss buffers are 1:1 with outstanding misses on the granularity of cache lines. So if you load 8 consecutive 4-byte values, which all fall in the same cache line, which misses in L1, you'll have 8 load buffer entries used, but only 1 fill buffer used since the buffer coalesces all misses to the same line.

BeeOnRope 8y ago

The simple answer is that these lanes (buffers, I prefer to call them) are per logical core. The more complicated answer is that there are a variety of buffers involved in the "path to memory", starting with the core-private buffers between the core and L1 and L2, and moving to socket-shared buffers at the L3, memory controllers and DRAM. In this particular single-threaded test, it is the 10 core-private "line fill buffers" between the L1 and L2 which are the bottleneck, which is where you get the simple answer from. If you ran this test on enough cores at the same time, I would entirely expect the bottlneck to change to one of the shared buffers.