4 ms·

That is quite an amazing way to put it. So the processor in my hand can compute a multiplication fast than light can cross the room?

jiggawatts 4y ago
It can complete many multiplications in that time, especially if you factor in parallelism. An 8-core machine using AVX-512 could do a few thousand 32-bit multiplications in that time. Your GPU can do tens of thousands, maybe hundreds of thousands depending on the model.
- bombcar 4y ago
  Or another way to look at it - the computer can do an absolute insane amount of math in the time it takes to roundtrip a single byte to the datacenter in US-West.
  - jiggawatts 4y ago
    The more generic way I like to put it is that throughput has been improving exponentially for decades thanks to Moore's law, but latency hasn't changed much at all and has a hard limit due to the speed of light. Hence the ratio between latency and compute has been changing exponentially. Even a linear or quadratic change would be dramatic, but exponential is something people just can't wrap their heads around. They're unable to really internalise it, in much the same way that in the early days of COVID people couldn't quite fathom how it is possible to go from 3-5 cases per day to tens of thousands. HDD random I/O latencies are about 10 to 100x slower than a network hope. These days local SSD latencies are about 100x better than a typical network hop and this is just going to keep going. It'll soon be 1,000x better, then 10,000x, etc... Any architecture using "remote storage" or "remote database calls" will be absolutely hamstrung by this. It'll be the equivalent of throwing away 99.99% or even 99.999% of the available performance. People will eventually wise up to this and start switching over to distributed databases that run in the same VM/container as the application tier. So instead of "N" web servers talking to "M" database servers, it'll be N+M nodes with both components deployed into them. Whatever argument can be made against this new architecture will become exponentially invalidated over time. Putting everything together is "too many GB of software to deploy"? Bzzt... we'll have 1 TB ram soon in typical servers. The CPU load of both together is too high? Bzzt.. the next EPYC CPUs will likely have 128 cores! Cache thrashing a problem? Bzzt... 1 GB and larger L3/L4 CPU caches are just around the corner.
    - mellavora 4y ago
      > slower than a network hope. Sometimes typos contain wisdom
  - Aeolun 4y ago
    Millions of multiplications in the time it takes me to open the ‘calc’ app.
    - jiggawatts 4y ago
      Billions. Your estimate is off by 3 orders of magnitude.
      - Aeolun 4y ago
        You underestimate the speed at which I open the app ;)
wruza 4y ago
Not so extraordinary if you take into account that CPU is essentially a thumb-sized labyrinth for light.
- orestarod 4y ago
  Not light. It's electric potential that moves around. No visible photons involved.

jiggawatts 4y ago

It can complete many multiplications in that time, especially if you factor in parallelism. An 8-core machine using AVX-512 could do a few thousand 32-bit multiplications in that time. Your GPU can do tens of thousands, maybe hundreds of thousands depending on the model.

bombcar 4y ago
Or another way to look at it - the computer can do an absolute insane amount of math in the time it takes to roundtrip a single byte to the datacenter in US-West.
- jiggawatts 4y ago
  The more generic way I like to put it is that throughput has been improving exponentially for decades thanks to Moore's law, but latency hasn't changed much at all and has a hard limit due to the speed of light. Hence the ratio between latency and compute has been changing exponentially. Even a linear or quadratic change would be dramatic, but exponential is something people just can't wrap their heads around. They're unable to really internalise it, in much the same way that in the early days of COVID people couldn't quite fathom how it is possible to go from 3-5 cases per day to tens of thousands. HDD random I/O latencies are about 10 to 100x slower than a network hope. These days local SSD latencies are about 100x better than a typical network hop and this is just going to keep going. It'll soon be 1,000x better, then 10,000x, etc... Any architecture using "remote storage" or "remote database calls" will be absolutely hamstrung by this. It'll be the equivalent of throwing away 99.99% or even 99.999% of the available performance. People will eventually wise up to this and start switching over to distributed databases that run in the same VM/container as the application tier. So instead of "N" web servers talking to "M" database servers, it'll be N+M nodes with both components deployed into them. Whatever argument can be made against this new architecture will become exponentially invalidated over time. Putting everything together is "too many GB of software to deploy"? Bzzt... we'll have 1 TB ram soon in typical servers. The CPU load of both together is too high? Bzzt.. the next EPYC CPUs will likely have 128 cores! Cache thrashing a problem? Bzzt... 1 GB and larger L3/L4 CPU caches are just around the corner.
  - mellavora 4y ago
    > slower than a network hope. Sometimes typos contain wisdom
- Aeolun 4y ago
  Millions of multiplications in the time it takes me to open the ‘calc’ app.
  - jiggawatts 4y ago
    Billions. Your estimate is off by 3 orders of magnitude.
    - Aeolun 4y ago
      You underestimate the speed at which I open the app ;)

wruza 4y ago

Not so extraordinary if you take into account that CPU is essentially a thumb-sized labyrinth for light.

orestarod 4y ago
Not light. It's electric potential that moves around. No visible photons involved.