6 ms·

> If you hold up a sign with, say, a multiplication, a CPU will produce the result before light reaches a person a few metres away. The latency on multiplicati

> If you hold up a sign with, say, a multiplication, a CPU will produce the result before light reaches a person a few metres away. The latency on multiplication (register input to register output) is 5-clock ticks, and many computers are 4GHz or 5GHz these days. 5-clock cycles at 5GHz is 1ns, which is 30-centimeters of light travel. If we include L1 cache read and L1 cache write, IIRC its 4 clock cycles for read + 4 more for the write. So 13 clock ticks, which is almost 70 centimeters. ------------ DDR4 read and L1 cache write will add 50 nanoseconds (~250 cycles) of delay, and we're up to 13 meters. And now you know why cache exists, otherwise computers will be waiting on DDR4 RAM all day, rather than doing work.

moonchild 4y ago
> The latency on multiplication (register input to register output) is 5-clock ticks 3 https://www.agner.org/optimize/instruction_tables.pdf https://www.agner.org/optimize/instruction_tables.pdf
- jnordwick 4y ago
  Those insrtruction latencies are in addition to the pipeline created latency. (They are actually the number of cycles added to the dependency chain specifically). The mult port has a small pipeline itself of 3 stages (that why 3 cycles latency). Intel has a 5 stage pipeline so the minimum latency is going to be 8 for just those two things.
  - gpderetta 4y ago
    I don't understand what you are trying to say. The dependency chain length is what is normally intended as instruction latency. Also the pipeline length is certainly not 5 stages but more like 20-30.
    - jnordwick 4y ago
      Sorry, I dropped all the 1s in that message when i typed it (laptop keyboard is a little sketchy right now). That should have been 15 and 18. I think the recent Intel microarchs take 14 plus howewever long at uopd decoding that minimum 1, so 15-20 or close to that. > The dependency chain length is what is normally intended as instruction latency. Yes, the way I read the original post and others was that you actually your response back in 3 cycles, which isn't correct. It doesn't get comitted for a while (but following instructions can use the result even if it hasn't been committed yet). You're not getting a result in less than 20 cycles basically.
      - gpderetta 4y ago
        > Sorry, I dropped all the 1s in that message when i typed it It makes sense now! :D > You're not getting a result in less than 20 cycles basically But the end of the pipeline is an arbitrary point. It will take a few more cycles to get to L1 (when it makes it out of the write buffer), a few tens more to traverse the L2 and L3 and hundreds to get to RAM (if it gets there at all). If it has to get to an human it will take thousands of cycles to get through the various busses to a screen or similar. The only reasonably useful latency measure is what it takes for the value to be ready to be consumed by the next instruction, which is indeed 3-5 cycles depending to the specific microarchitecture.
        jnordwick 4y ago
        > which is indeed 3-5 cycles depending to the specific microarchitecture I assume you are talking about from fetch to hitting the store buffer? That would be the aabsolute min time before the data could be seen elsehwere I would think. It can still potentially be rolled back, and that would be higher than reciprocal be way too fast to sustain, but for a single instr burst, I'm not sure. So much happens at the same time, An L1 read hit will cost you 4 minimum, hut all but 1 of that is hidden. can't avoid the multi cost of 3 or add 1. the decoding and uop cache hit, reservation, etc will cost a few. I have no idea. If you know of anything describing it in such detail, I would be comopletely curiouis.
- jiggawatts 4y ago
  Back in the days, an integer division took something like 46 clocks (original Pentium), and now on Ice Lake it's just 12 with a reciprocal throughput of 6. Multiply that by the clock speed increase and a modern CPU can "do division" about 300-400 times faster than a Pentium could. Then multiply that by the number of cores available now versus just one core, and that increases to about 2000 times faster! I used to play 3D games on Pentium-based machines and I thought of them as a "huge upgrade" from 486, which in turn were a huge upgrade from 286, etc... Now, people with Ice Lake CPUs in their laptops and servers complain that things are slow.
  - bouke 4y ago
    And things are slow as we waste all that processing power on running javascript one way or another. And everything requires a slow blocking connection to the mainframe. Nowadays the “always connected” mindset is really slowing us down.
    - onion2k 4y ago
      That explains why Electron apps and Web pages are slow but the post you're replying to is about games...
      - bouke 4y ago
        Ok fair enough, but the mindset is spreading: json (javascript) parsing is what caused GTA Online loading times to balloon and I dread playing Call of Duty online as it wants to download and install dozens of GBs every time I launch it.
        wruza 4y ago
        It wasn’t json parsing per se, but a buggy roll-your-own implementation that used a function (sscanf iirc) with a surprising nontrivial complexity on a long string. Fun part is, if they just outsourced that load to javascript and its JSON.parse, they’d never encounter that exponential slowdown. Javascript is a nice target to blame, but it isn’t the problem. CPUs got hundreds of times faster, javascript only divides it by N, which stays low and constant (at least) through decades. Do you really believe that if browsers only supported MSVC++-based DLLs without scripting, sites would run faster? That would be naive.
reacharavindh 4y ago
This reminds me of that “todo” I wrote for myself a long time ago. These days processors come with bigger L1,L2, and L3 caches. Would it be possible for a program that works on a tiny bit of data(few KB) to load it all up in the cache and provide ultimate response times?! Are there any directives to the Operating System to say - “here keep this data in the fastest accessible L[1,2,3] please”?
- l33t2328 4y ago
  I don’t think you can generally control the cache with such granularity since modern processors do all sorts of instruction level parallelism and cache coherency voodoo
- deredede 4y ago
  On CPUs you can't really force data to stay on the cache, but if you access it frequently and there is not too much load, it will stay there anyways. Some architectures (e.g. GPUs) provide local "scratchpad" memories instead of (or in addition to) caches. These are separate uninitialized adressable memory region with similar access times to a L2/L1 cache.
- 2143 4y ago
  > Are there any directives to the Operating System to say - “here keep this data in the fastest accessible L[1,2,3] please”? I'm probably the worst person to explain this. Long long ago, I took a parallel programming class in grad school. It turns out the conventional way to do matrix multiplication results in plenty of cache misses. However, if you carefully tweak the order of the loops and do certain minor modifications — I forget the details — you could substantially increase the cache hits and make matrix multiplication go noticeably faster on benchmarks. Some random details that may be relevant: * When the processor loads a single number M[x][y], it sort of loads in the adjacent numbers as well. You need to take advantage of this. * Something about row-major/column-major array is an important detail. What I'm trying to say is, it is possible to indirectly optimize cache hits by careful manual hand tweaking. I don't know if there's a general automagic way to do this though. This probably wasn't very useful, but I'm just putting it out there. Maybe more knowledgeable folks can explain this better.

moonchild 4y ago

> The latency on multiplication (register input to register output) is 5-clock ticks 3 https://www.agner.org/optimize/instruction_tables.pdf https://www.agner.org/optimize/instruction_tables.pdf

jnordwick 4y ago
Those insrtruction latencies are in addition to the pipeline created latency. (They are actually the number of cycles added to the dependency chain specifically). The mult port has a small pipeline itself of 3 stages (that why 3 cycles latency). Intel has a 5 stage pipeline so the minimum latency is going to be 8 for just those two things.
- gpderetta 4y ago
  I don't understand what you are trying to say. The dependency chain length is what is normally intended as instruction latency. Also the pipeline length is certainly not 5 stages but more like 20-30.
  - jnordwick 4y ago
    Sorry, I dropped all the 1s in that message when i typed it (laptop keyboard is a little sketchy right now). That should have been 15 and 18. I think the recent Intel microarchs take 14 plus howewever long at uopd decoding that minimum 1, so 15-20 or close to that. > The dependency chain length is what is normally intended as instruction latency. Yes, the way I read the original post and others was that you actually your response back in 3 cycles, which isn't correct. It doesn't get comitted for a while (but following instructions can use the result even if it hasn't been committed yet). You're not getting a result in less than 20 cycles basically.
    - gpderetta 4y ago
      > Sorry, I dropped all the 1s in that message when i typed it It makes sense now! :D > You're not getting a result in less than 20 cycles basically But the end of the pipeline is an arbitrary point. It will take a few more cycles to get to L1 (when it makes it out of the write buffer), a few tens more to traverse the L2 and L3 and hundreds to get to RAM (if it gets there at all). If it has to get to an human it will take thousands of cycles to get through the various busses to a screen or similar. The only reasonably useful latency measure is what it takes for the value to be ready to be consumed by the next instruction, which is indeed 3-5 cycles depending to the specific microarchitecture.
      - jnordwick 4y ago
        > which is indeed 3-5 cycles depending to the specific microarchitecture I assume you are talking about from fetch to hitting the store buffer? That would be the aabsolute min time before the data could be seen elsehwere I would think. It can still potentially be rolled back, and that would be higher than reciprocal be way too fast to sustain, but for a single instr burst, I'm not sure. So much happens at the same time, An L1 read hit will cost you 4 minimum, hut all but 1 of that is hidden. can't avoid the multi cost of 3 or add 1. the decoding and uop cache hit, reservation, etc will cost a few. I have no idea. If you know of anything describing it in such detail, I would be comopletely curiouis.
jiggawatts 4y ago
Back in the days, an integer division took something like 46 clocks (original Pentium), and now on Ice Lake it's just 12 with a reciprocal throughput of 6. Multiply that by the clock speed increase and a modern CPU can "do division" about 300-400 times faster than a Pentium could. Then multiply that by the number of cores available now versus just one core, and that increases to about 2000 times faster! I used to play 3D games on Pentium-based machines and I thought of them as a "huge upgrade" from 486, which in turn were a huge upgrade from 286, etc... Now, people with Ice Lake CPUs in their laptops and servers complain that things are slow.
- bouke 4y ago
  And things are slow as we waste all that processing power on running javascript one way or another. And everything requires a slow blocking connection to the mainframe. Nowadays the “always connected” mindset is really slowing us down.
  - onion2k 4y ago
    That explains why Electron apps and Web pages are slow but the post you're replying to is about games...
    - bouke 4y ago
      Ok fair enough, but the mindset is spreading: json (javascript) parsing is what caused GTA Online loading times to balloon and I dread playing Call of Duty online as it wants to download and install dozens of GBs every time I launch it.
      - wruza 4y ago
        It wasn’t json parsing per se, but a buggy roll-your-own implementation that used a function (sscanf iirc) with a surprising nontrivial complexity on a long string. Fun part is, if they just outsourced that load to javascript and its JSON.parse, they’d never encounter that exponential slowdown. Javascript is a nice target to blame, but it isn’t the problem. CPUs got hundreds of times faster, javascript only divides it by N, which stays low and constant (at least) through decades. Do you really believe that if browsers only supported MSVC++-based DLLs without scripting, sites would run faster? That would be naive.

reacharavindh 4y ago

This reminds me of that “todo” I wrote for myself a long time ago. These days processors come with bigger L1,L2, and L3 caches. Would it be possible for a program that works on a tiny bit of data(few KB) to load it all up in the cache and provide ultimate response times?! Are there any directives to the Operating System to say - “here keep this data in the fastest accessible L[1,2,3] please”?

l33t2328 4y ago
I don’t think you can generally control the cache with such granularity since modern processors do all sorts of instruction level parallelism and cache coherency voodoo
deredede 4y ago
On CPUs you can't really force data to stay on the cache, but if you access it frequently and there is not too much load, it will stay there anyways. Some architectures (e.g. GPUs) provide local "scratchpad" memories instead of (or in addition to) caches. These are separate uninitialized adressable memory region with similar access times to a L2/L1 cache.
2143 4y ago
> Are there any directives to the Operating System to say - “here keep this data in the fastest accessible L[1,2,3] please”? I'm probably the worst person to explain this. Long long ago, I took a parallel programming class in grad school. It turns out the conventional way to do matrix multiplication results in plenty of cache misses. However, if you carefully tweak the order of the loops and do certain minor modifications — I forget the details — you could substantially increase the cache hits and make matrix multiplication go noticeably faster on benchmarks. Some random details that may be relevant: * When the processor loads a single number M[x][y], it sort of loads in the adjacent numbers as well. You need to take advantage of this. * Something about row-major/column-major array is an important detail. What I'm trying to say is, it is possible to indirectly optimize cache hits by careful manual hand tweaking. I don't know if there's a general automagic way to do this though. This probably wasn't very useful, but I'm just putting it out there. Maybe more knowledgeable folks can explain this better.