10 ms·

On a 3GHz CPU, one clock cycle is enough time for light to travel only 10cm. If you hold up a sign with, say, a multiplication, a CPU will produce the result b

On a 3GHz CPU, one clock cycle is enough time for light to travel only 10cm. If you hold up a sign with, say, a multiplication, a CPU will produce the result before light reaches a person a few metres away.

dragontamer 4y ago
> If you hold up a sign with, say, a multiplication, a CPU will produce the result before light reaches a person a few metres away. The latency on multiplication (register input to register output) is 5-clock ticks, and many computers are 4GHz or 5GHz these days. 5-clock cycles at 5GHz is 1ns, which is 30-centimeters of light travel. If we include L1 cache read and L1 cache write, IIRC its 4 clock cycles for read + 4 more for the write. So 13 clock ticks, which is almost 70 centimeters. ------------ DDR4 read and L1 cache write will add 50 nanoseconds (~250 cycles) of delay, and we're up to 13 meters. And now you know why cache exists, otherwise computers will be waiting on DDR4 RAM all day, rather than doing work.
- moonchild 4y ago
  > The latency on multiplication (register input to register output) is 5-clock ticks 3 https://www.agner.org/optimize/instruction_tables.pdf https://www.agner.org/optimize/instruction_tables.pdf
  - jnordwick 4y ago
    Those insrtruction latencies are in addition to the pipeline created latency. (They are actually the number of cycles added to the dependency chain specifically). The mult port has a small pipeline itself of 3 stages (that why 3 cycles latency). Intel has a 5 stage pipeline so the minimum latency is going to be 8 for just those two things.
    - gpderetta 4y ago
      I don't understand what you are trying to say. The dependency chain length is what is normally intended as instruction latency. Also the pipeline length is certainly not 5 stages but more like 20-30.
      - jnordwick 4y ago
        Sorry, I dropped all the 1s in that message when i typed it (laptop keyboard is a little sketchy right now). That should have been 15 and 18. I think the recent Intel microarchs take 14 plus howewever long at uopd decoding that minimum 1, so 15-20 or close to that. > The dependency chain length is what is normally intended as instruction latency. Yes, the way I read the original post and others was that you actually your response back in 3 cycles, which isn't correct. It doesn't get comitted for a while (but following instructions can use the result even if it hasn't been committed yet). You're not getting a result in less than 20 cycles basically.
        gpderetta 4y ago
        > Sorry, I dropped all the 1s in that message when i typed it It makes sense now! :D > You're not getting a result in less than 20 cycles basically But the end of the pipeline is an arbitrary point. It will take a few more cycles to get to L1 (when it makes it out of the write buffer), a few tens more to traverse the L2 and L3 and hundreds to get to RAM (if it gets there at all). If it has to get to an human it will take thousands of cycles to get through the various busses to a screen or similar. The only reasonably useful latency measure is what it takes for the value to be ready to be consumed by the next instruction, which is indeed 3-5 cycles depending to the specific microarchitecture.
        jnordwick 4y ago
        > which is indeed 3-5 cycles depending to the specific microarchitecture I assume you are talking about from fetch to hitting the store buffer? That would be the aabsolute min time before the data could be seen elsehwere I would think. It can still potentially be rolled back, and that would be higher than reciprocal be way too fast to sustain, but for a single instr burst, I'm not sure. So much happens at the same time, An L1 read hit will cost you 4 minimum, hut all but 1 of that is hidden. can't avoid the multi cost of 3 or add 1. the decoding and uop cache hit, reservation, etc will cost a few. I have no idea. If you know of anything describing it in such detail, I would be comopletely curiouis.
  - jiggawatts 4y ago
    Back in the days, an integer division took something like 46 clocks (original Pentium), and now on Ice Lake it's just 12 with a reciprocal throughput of 6. Multiply that by the clock speed increase and a modern CPU can "do division" about 300-400 times faster than a Pentium could. Then multiply that by the number of cores available now versus just one core, and that increases to about 2000 times faster! I used to play 3D games on Pentium-based machines and I thought of them as a "huge upgrade" from 486, which in turn were a huge upgrade from 286, etc... Now, people with Ice Lake CPUs in their laptops and servers complain that things are slow.
    - bouke 4y ago
      And things are slow as we waste all that processing power on running javascript one way or another. And everything requires a slow blocking connection to the mainframe. Nowadays the “always connected” mindset is really slowing us down.
      - onion2k 4y ago
        That explains why Electron apps and Web pages are slow but the post you're replying to is about games...
        bouke 4y ago
        Ok fair enough, but the mindset is spreading: json (javascript) parsing is what caused GTA Online loading times to balloon and I dread playing Call of Duty online as it wants to download and install dozens of GBs every time I launch it.
        wruza 4y ago
        It wasn’t json parsing per se, but a buggy roll-your-own implementation that used a function (sscanf iirc) with a surprising nontrivial complexity on a long string. Fun part is, if they just outsourced that load to javascript and its JSON.parse, they’d never encounter that exponential slowdown. Javascript is a nice target to blame, but it isn’t the problem. CPUs got hundreds of times faster, javascript only divides it by N, which stays low and constant (at least) through decades. Do you really believe that if browsers only supported MSVC++-based DLLs without scripting, sites would run faster? That would be naive.
- reacharavindh 4y ago
  This reminds me of that “todo” I wrote for myself a long time ago. These days processors come with bigger L1,L2, and L3 caches. Would it be possible for a program that works on a tiny bit of data(few KB) to load it all up in the cache and provide ultimate response times?! Are there any directives to the Operating System to say - “here keep this data in the fastest accessible L[1,2,3] please”?
  - l33t2328 4y ago
    I don’t think you can generally control the cache with such granularity since modern processors do all sorts of instruction level parallelism and cache coherency voodoo
  - deredede 4y ago
    On CPUs you can't really force data to stay on the cache, but if you access it frequently and there is not too much load, it will stay there anyways. Some architectures (e.g. GPUs) provide local "scratchpad" memories instead of (or in addition to) caches. These are separate uninitialized adressable memory region with similar access times to a L2/L1 cache.
  - 2143 4y ago
    > Are there any directives to the Operating System to say - “here keep this data in the fastest accessible L[1,2,3] please”? I'm probably the worst person to explain this. Long long ago, I took a parallel programming class in grad school. It turns out the conventional way to do matrix multiplication results in plenty of cache misses. However, if you carefully tweak the order of the loops and do certain minor modifications — I forget the details — you could substantially increase the cache hits and make matrix multiplication go noticeably faster on benchmarks. Some random details that may be relevant: * When the processor loads a single number M[x][y], it sort of loads in the adjacent numbers as well. You need to take advantage of this. * Something about row-major/column-major array is an important detail. What I'm trying to say is, it is possible to indirectly optimize cache hits by careful manual hand tweaking. I don't know if there's a general automagic way to do this though. This probably wasn't very useful, but I'm just putting it out there. Maybe more knowledgeable folks can explain this better.
diroussel 4y ago
That is quite an amazing way to put it. So the processor in my hand can compute a multiplication fast than light can cross the room?
- jiggawatts 4y ago
  It can complete many multiplications in that time, especially if you factor in parallelism. An 8-core machine using AVX-512 could do a few thousand 32-bit multiplications in that time. Your GPU can do tens of thousands, maybe hundreds of thousands depending on the model.
  - bombcar 4y ago
    Or another way to look at it - the computer can do an absolute insane amount of math in the time it takes to roundtrip a single byte to the datacenter in US-West.
    - jiggawatts 4y ago
      The more generic way I like to put it is that throughput has been improving exponentially for decades thanks to Moore's law, but latency hasn't changed much at all and has a hard limit due to the speed of light. Hence the ratio between latency and compute has been changing exponentially. Even a linear or quadratic change would be dramatic, but exponential is something people just can't wrap their heads around. They're unable to really internalise it, in much the same way that in the early days of COVID people couldn't quite fathom how it is possible to go from 3-5 cases per day to tens of thousands. HDD random I/O latencies are about 10 to 100x slower than a network hope. These days local SSD latencies are about 100x better than a typical network hop and this is just going to keep going. It'll soon be 1,000x better, then 10,000x, etc... Any architecture using "remote storage" or "remote database calls" will be absolutely hamstrung by this. It'll be the equivalent of throwing away 99.99% or even 99.999% of the available performance. People will eventually wise up to this and start switching over to distributed databases that run in the same VM/container as the application tier. So instead of "N" web servers talking to "M" database servers, it'll be N+M nodes with both components deployed into them. Whatever argument can be made against this new architecture will become exponentially invalidated over time. Putting everything together is "too many GB of software to deploy"? Bzzt... we'll have 1 TB ram soon in typical servers. The CPU load of both together is too high? Bzzt.. the next EPYC CPUs will likely have 128 cores! Cache thrashing a problem? Bzzt... 1 GB and larger L3/L4 CPU caches are just around the corner.
grishka 4y ago
Huh, but then I'm pretty sure that there are some paths inside the CPU die that are long enough that speed of light is a consideration at these frequencies. Must require a lot of smart people to design these things, yet it only takes a bunch of junior developers to bog them down.
- google234123 4y ago
  I say we ban all junior devs.
  - morelisp 4y ago
    Easier to raise the speed of light.
    - salawat 4y ago
      And risk the universal UB? No thanks.
- orestarod 4y ago
  There is indeed the speed of propagation of electric potential taken into account, that is, how long it takes for the input of a logical gate or a logical subsystem to produce the output (that involves the propagation of electric potential through the chip's conductors). If your clock is too fast for the size of your subsystem, the result will not be correct at the output before the next cycle begins, so your system will just be bogus.
generalizations 4y ago
I ran across an animation once that showed graphically the time it takes light to travel between the planets and the sun. It's weird, but light doesn't seem that fast anymore.
- shultays 4y ago
  I feel like it is more like we cant comprehend how big and empty space is
- edbaskerville 4y ago
  The thing that did for me is realizing that people on opposite sides of the United States can't play music together if it requires any rhythmic coordination, even with a true speed-of-light signal with no other sources of latency.
  - aeonik 4y ago
    jamtaba and ninjam are an open source solutions to this problem. They allow you to buffer everyone's playing, at a user specified interval, then replays the last measure of music to everyone. It's definitely not the same as live playing, but it's still pretty fun, and actually forces you to get creative on different ways. https://jamtaba-music-web-site.appspot.com/ https://jamtaba-music-web-site.appspot.com/
    - Aeolun 4y ago
      How does that work for the one playing ahead of everyone else? He just doesn’t hear anything? Or he hears his own music from 1 second ago? Or worse, other people’s music from 1 second ago.
      - rocqua 4y ago
        I think everyone hears everyone else playing x milliseconds ago? Where x is probably decided based on the beat. The weird side-effects are why I guess the parent said that "it's still pretty fun, and actually forces you to get creative on different ways."
dekhn 4y ago
Eventually you can't stuff enough computing in a small area (power density). Therefore you have to connect multiple CPUs spread out in space. The limit for many supercomputers is about how long it takes light or electrical signals to travel about 20 meters. Latency to first result is only part of the measurement that matters.
capableweb 4y ago
If you're using a normal monitor, the bottleneck would be transferring the results of the calculation to the monitor, which commonly have a latency of 3ms or more. So when the monitor displays the calculations, the CPU has already moved on to other things :)
- drewtato 4y ago
  It takes 10-20ms for the pixels to transition on an LCD display. And on 60Hz, it's 8+/-8ms for the monitor to actually address the row with your information. Luckily, the CPU doesn't need to wait for the monitor. And the slowest part of the chain will almost always be getting it from your eyes to your hands (250ms+).
denzil 4y ago
This comparison reminds me of Grace Hopper explaining how long nanosecond is: https://www.youtube.com/watch?v=9eyFDBPk4Yw https://www.youtube.com/watch?v=9eyFDBPk4Yw

dragontamer 4y ago

> If you hold up a sign with, say, a multiplication, a CPU will produce the result before light reaches a person a few metres away. The latency on multiplication (register input to register output) is 5-clock ticks, and many computers are 4GHz or 5GHz these days. 5-clock cycles at 5GHz is 1ns, which is 30-centimeters of light travel. If we include L1 cache read and L1 cache write, IIRC its 4 clock cycles for read + 4 more for the write. So 13 clock ticks, which is almost 70 centimeters. ------------ DDR4 read and L1 cache write will add 50 nanoseconds (~250 cycles) of delay, and we're up to 13 meters. And now you know why cache exists, otherwise computers will be waiting on DDR4 RAM all day, rather than doing work.

moonchild 4y ago
> The latency on multiplication (register input to register output) is 5-clock ticks 3 https://www.agner.org/optimize/instruction_tables.pdf https://www.agner.org/optimize/instruction_tables.pdf
- jnordwick 4y ago
  Those insrtruction latencies are in addition to the pipeline created latency. (They are actually the number of cycles added to the dependency chain specifically). The mult port has a small pipeline itself of 3 stages (that why 3 cycles latency). Intel has a 5 stage pipeline so the minimum latency is going to be 8 for just those two things.
  - gpderetta 4y ago
    I don't understand what you are trying to say. The dependency chain length is what is normally intended as instruction latency. Also the pipeline length is certainly not 5 stages but more like 20-30.
    - jnordwick 4y ago
      Sorry, I dropped all the 1s in that message when i typed it (laptop keyboard is a little sketchy right now). That should have been 15 and 18. I think the recent Intel microarchs take 14 plus howewever long at uopd decoding that minimum 1, so 15-20 or close to that. > The dependency chain length is what is normally intended as instruction latency. Yes, the way I read the original post and others was that you actually your response back in 3 cycles, which isn't correct. It doesn't get comitted for a while (but following instructions can use the result even if it hasn't been committed yet). You're not getting a result in less than 20 cycles basically.
      - gpderetta 4y ago
        > Sorry, I dropped all the 1s in that message when i typed it It makes sense now! :D > You're not getting a result in less than 20 cycles basically But the end of the pipeline is an arbitrary point. It will take a few more cycles to get to L1 (when it makes it out of the write buffer), a few tens more to traverse the L2 and L3 and hundreds to get to RAM (if it gets there at all). If it has to get to an human it will take thousands of cycles to get through the various busses to a screen or similar. The only reasonably useful latency measure is what it takes for the value to be ready to be consumed by the next instruction, which is indeed 3-5 cycles depending to the specific microarchitecture.
        jnordwick 4y ago
        > which is indeed 3-5 cycles depending to the specific microarchitecture I assume you are talking about from fetch to hitting the store buffer? That would be the aabsolute min time before the data could be seen elsehwere I would think. It can still potentially be rolled back, and that would be higher than reciprocal be way too fast to sustain, but for a single instr burst, I'm not sure. So much happens at the same time, An L1 read hit will cost you 4 minimum, hut all but 1 of that is hidden. can't avoid the multi cost of 3 or add 1. the decoding and uop cache hit, reservation, etc will cost a few. I have no idea. If you know of anything describing it in such detail, I would be comopletely curiouis.
- jiggawatts 4y ago
  Back in the days, an integer division took something like 46 clocks (original Pentium), and now on Ice Lake it's just 12 with a reciprocal throughput of 6. Multiply that by the clock speed increase and a modern CPU can "do division" about 300-400 times faster than a Pentium could. Then multiply that by the number of cores available now versus just one core, and that increases to about 2000 times faster! I used to play 3D games on Pentium-based machines and I thought of them as a "huge upgrade" from 486, which in turn were a huge upgrade from 286, etc... Now, people with Ice Lake CPUs in their laptops and servers complain that things are slow.
  - bouke 4y ago
    And things are slow as we waste all that processing power on running javascript one way or another. And everything requires a slow blocking connection to the mainframe. Nowadays the “always connected” mindset is really slowing us down.
    - onion2k 4y ago
      That explains why Electron apps and Web pages are slow but the post you're replying to is about games...
      - bouke 4y ago
        Ok fair enough, but the mindset is spreading: json (javascript) parsing is what caused GTA Online loading times to balloon and I dread playing Call of Duty online as it wants to download and install dozens of GBs every time I launch it.
        wruza 4y ago
        It wasn’t json parsing per se, but a buggy roll-your-own implementation that used a function (sscanf iirc) with a surprising nontrivial complexity on a long string. Fun part is, if they just outsourced that load to javascript and its JSON.parse, they’d never encounter that exponential slowdown. Javascript is a nice target to blame, but it isn’t the problem. CPUs got hundreds of times faster, javascript only divides it by N, which stays low and constant (at least) through decades. Do you really believe that if browsers only supported MSVC++-based DLLs without scripting, sites would run faster? That would be naive.
reacharavindh 4y ago
This reminds me of that “todo” I wrote for myself a long time ago. These days processors come with bigger L1,L2, and L3 caches. Would it be possible for a program that works on a tiny bit of data(few KB) to load it all up in the cache and provide ultimate response times?! Are there any directives to the Operating System to say - “here keep this data in the fastest accessible L[1,2,3] please”?
- l33t2328 4y ago
  I don’t think you can generally control the cache with such granularity since modern processors do all sorts of instruction level parallelism and cache coherency voodoo
- deredede 4y ago
  On CPUs you can't really force data to stay on the cache, but if you access it frequently and there is not too much load, it will stay there anyways. Some architectures (e.g. GPUs) provide local "scratchpad" memories instead of (or in addition to) caches. These are separate uninitialized adressable memory region with similar access times to a L2/L1 cache.
- 2143 4y ago
  > Are there any directives to the Operating System to say - “here keep this data in the fastest accessible L[1,2,3] please”? I'm probably the worst person to explain this. Long long ago, I took a parallel programming class in grad school. It turns out the conventional way to do matrix multiplication results in plenty of cache misses. However, if you carefully tweak the order of the loops and do certain minor modifications — I forget the details — you could substantially increase the cache hits and make matrix multiplication go noticeably faster on benchmarks. Some random details that may be relevant: * When the processor loads a single number M[x][y], it sort of loads in the adjacent numbers as well. You need to take advantage of this. * Something about row-major/column-major array is an important detail. What I'm trying to say is, it is possible to indirectly optimize cache hits by careful manual hand tweaking. I don't know if there's a general automagic way to do this though. This probably wasn't very useful, but I'm just putting it out there. Maybe more knowledgeable folks can explain this better.

diroussel 4y ago

That is quite an amazing way to put it. So the processor in my hand can compute a multiplication fast than light can cross the room?

jiggawatts 4y ago
It can complete many multiplications in that time, especially if you factor in parallelism. An 8-core machine using AVX-512 could do a few thousand 32-bit multiplications in that time. Your GPU can do tens of thousands, maybe hundreds of thousands depending on the model.
- bombcar 4y ago
  Or another way to look at it - the computer can do an absolute insane amount of math in the time it takes to roundtrip a single byte to the datacenter in US-West.
  - jiggawatts 4y ago
    The more generic way I like to put it is that throughput has been improving exponentially for decades thanks to Moore's law, but latency hasn't changed much at all and has a hard limit due to the speed of light. Hence the ratio between latency and compute has been changing exponentially. Even a linear or quadratic change would be dramatic, but exponential is something people just can't wrap their heads around. They're unable to really internalise it, in much the same way that in the early days of COVID people couldn't quite fathom how it is possible to go from 3-5 cases per day to tens of thousands. HDD random I/O latencies are about 10 to 100x slower than a network hope. These days local SSD latencies are about 100x better than a typical network hop and this is just going to keep going. It'll soon be 1,000x better, then 10,000x, etc... Any architecture using "remote storage" or "remote database calls" will be absolutely hamstrung by this. It'll be the equivalent of throwing away 99.99% or even 99.999% of the available performance. People will eventually wise up to this and start switching over to distributed databases that run in the same VM/container as the application tier. So instead of "N" web servers talking to "M" database servers, it'll be N+M nodes with both components deployed into them. Whatever argument can be made against this new architecture will become exponentially invalidated over time. Putting everything together is "too many GB of software to deploy"? Bzzt... we'll have 1 TB ram soon in typical servers. The CPU load of both together is too high? Bzzt.. the next EPYC CPUs will likely have 128 cores! Cache thrashing a problem? Bzzt... 1 GB and larger L3/L4 CPU caches are just around the corner.

grishka 4y ago

Huh, but then I'm pretty sure that there are some paths inside the CPU die that are long enough that speed of light is a consideration at these frequencies. Must require a lot of smart people to design these things, yet it only takes a bunch of junior developers to bog them down.

google234123 4y ago
I say we ban all junior devs.
- morelisp 4y ago
  Easier to raise the speed of light.
  - salawat 4y ago
    And risk the universal UB? No thanks.
orestarod 4y ago
There is indeed the speed of propagation of electric potential taken into account, that is, how long it takes for the input of a logical gate or a logical subsystem to produce the output (that involves the propagation of electric potential through the chip's conductors). If your clock is too fast for the size of your subsystem, the result will not be correct at the output before the next cycle begins, so your system will just be bogus.

generalizations 4y ago

I ran across an animation once that showed graphically the time it takes light to travel between the planets and the sun. It's weird, but light doesn't seem that fast anymore.

shultays 4y ago
I feel like it is more like we cant comprehend how big and empty space is
edbaskerville 4y ago
The thing that did for me is realizing that people on opposite sides of the United States can't play music together if it requires any rhythmic coordination, even with a true speed-of-light signal with no other sources of latency.
- aeonik 4y ago
  jamtaba and ninjam are an open source solutions to this problem. They allow you to buffer everyone's playing, at a user specified interval, then replays the last measure of music to everyone. It's definitely not the same as live playing, but it's still pretty fun, and actually forces you to get creative on different ways. https://jamtaba-music-web-site.appspot.com/ https://jamtaba-music-web-site.appspot.com/
  - Aeolun 4y ago
    How does that work for the one playing ahead of everyone else? He just doesn’t hear anything? Or he hears his own music from 1 second ago? Or worse, other people’s music from 1 second ago.
    - rocqua 4y ago
      I think everyone hears everyone else playing x milliseconds ago? Where x is probably decided based on the beat. The weird side-effects are why I guess the parent said that "it's still pretty fun, and actually forces you to get creative on different ways."

dekhn 4y ago

Eventually you can't stuff enough computing in a small area (power density). Therefore you have to connect multiple CPUs spread out in space. The limit for many supercomputers is about how long it takes light or electrical signals to travel about 20 meters. Latency to first result is only part of the measurement that matters.

capableweb 4y ago

If you're using a normal monitor, the bottleneck would be transferring the results of the calculation to the monitor, which commonly have a latency of 3ms or more. So when the monitor displays the calculations, the CPU has already moved on to other things :)

drewtato 4y ago
It takes 10-20ms for the pixels to transition on an LCD display. And on 60Hz, it's 8+/-8ms for the monitor to actually address the row with your information. Luckily, the CPU doesn't need to wait for the monitor. And the slowest part of the chain will almost always be getting it from your eyes to your hands (250ms+).

denzil 4y ago

This comparison reminds me of Grace Hopper explaining how long nanosecond is: https://www.youtube.com/watch?v=9eyFDBPk4Yw https://www.youtube.com/watch?v=9eyFDBPk4Yw