3 ms·

Apologies in advance for the layperson question: It's my understanding that a lot of CPU gains come from caching. That suggests to me that there is potential p

Apologies in advance for the layperson question: It's my understanding that a lot of CPU gains come from caching. That suggests to me that there is potential performance to be gained by caching across a larger number of machines. Is that something Amazon could do here? Somehow connect all their machines and cache in a huge space? Maybe individual physical machines would be more like a front end for a cache space, and when I get an Amazon instance, it's actually a "virtual" CPU that pieces together instructions that are mostly already cached in various places throughout the network? Is that even theoretically possible, or is it total fantasy?

rblatz 8y ago
Cache is insanely fast, orders of magnitude faster than ram, and basically instant compared to going to disk or another machine on the network. I would find it unlikely that they could overcome the added network latency introduced in such a system. Edit: check this out for more info https://people.eecs.berkeley.edu/~rcs/research/interactive_latency.html https://people.eecs.berkeley.edu/~rcs/research/interactive_l...
- sharpneli 8y ago
  And to give the 1ns L1 access time some physicality. During 1 nanosecond light in vacuum travels 30 centimeters, or 12 inches. Signals in conductors travel slower. It is ridiculously short amount of time. This means that there will absolutely never* be anything that can give faster access that is going to be farther than that from the CPU. Or more specifically half of that distance as the message to request what part of the cache to read must travel to the cache itself. * Unless we find out FTL is possible. But it's a rather safe bet to assume no.
Merad 8y ago
I'm going by memory here, but IIRC the fastest level of CPU cache (L1) can be accessed in about 2 nanoseconds. Even the slowest level (L3) operates on the order of a few hundred nanoseconds. RAM is slow by comparison, and network latency is so much slower it's basically glacial.
evancox100 8y ago
Your idea is impractical from a "caching for performance" perspective, for the reasons other have pointed out. However, there is work to enable multi-machine coherent memory. I just saw [1] today. This is kind of similar, you're giving a single, unified view to memory across multiple machines. But you do this to easily share data as part of a new programming paradigm, not to speed up your code. https://www.westerndigital.com/company/newsroom/press-releases/2018/2018-12-04-western-digital-delivers-new-innovations-to-drive-open-standard-interfaces-and-risc-v-processor-development https://www.westerndigital.com/company/newsroom/press-releas...

rblatz 8y ago

Cache is insanely fast, orders of magnitude faster than ram, and basically instant compared to going to disk or another machine on the network. I would find it unlikely that they could overcome the added network latency introduced in such a system. Edit: check this out for more info https://people.eecs.berkeley.edu/~rcs/research/interactive_latency.html https://people.eecs.berkeley.edu/~rcs/research/interactive_l...

sharpneli 8y ago
And to give the 1ns L1 access time some physicality. During 1 nanosecond light in vacuum travels 30 centimeters, or 12 inches. Signals in conductors travel slower. It is ridiculously short amount of time. This means that there will absolutely never* be anything that can give faster access that is going to be farther than that from the CPU. Or more specifically half of that distance as the message to request what part of the cache to read must travel to the cache itself. * Unless we find out FTL is possible. But it's a rather safe bet to assume no.

Merad 8y ago

I'm going by memory here, but IIRC the fastest level of CPU cache (L1) can be accessed in about 2 nanoseconds. Even the slowest level (L3) operates on the order of a few hundred nanoseconds. RAM is slow by comparison, and network latency is so much slower it's basically glacial.

evancox100 8y ago

Your idea is impractical from a "caching for performance" perspective, for the reasons other have pointed out. However, there is work to enable multi-machine coherent memory. I just saw [1] today. This is kind of similar, you're giving a single, unified view to memory across multiple machines. But you do this to easily share data as part of a new programming paradigm, not to speed up your code. https://www.westerndigital.com/company/newsroom/press-releases/2018/2018-12-04-western-digital-delivers-new-innovations-to-drive-open-standard-interfaces-and-risc-v-processor-development https://www.westerndigital.com/company/newsroom/press-releas...