4 ms·

More on Apple’s A9X SoC: 147mm2TSMC, 12 GPU Cores, No L3 Cache

30 points by lelf 11y ago

Analemma_ 11y ago
The complete lack of an L3 cache is fascinating if true. Not being an expert on chip design, I had thought the "number of cache levels" trend was monotonically increasing, especially since some of Intel's latest chips now have L4 caches, but now this. Can someone explain how they can get away with this? Is it enough to just have a really fast and wide bus, did they possibly need to make changes on the OS level, or is it just more acceptable on a device where you'll only be running one or two applications and very few background daemons/services?
- stygiansonic 11y ago
  The article had this to say: "One explanation may be that Apple deemed the L3 cache no longer necessary with the A9X’s 128-bit LPDDR4 memory bus; that 51.2GB/sec of bandwidth meant that they no longer needed the cache to avoid GPU stalls." And: "Our own Andrei Frumusanu suspects that it may be a power matter, and that Apple was using the L3 cache to save on power-expensive memory operations on the A9. With A9X however, it’s a tablet SoC that doesn’t face the same power restrictions, and as a result doesn’t need a power-saving cache. This would be coupled with the fact that with double the GPU cores, there would be a lot more pressure on just a 4MB cache versus the pressure created by A9, which in turn may drive the need for a larger cache and ultimately an even larger die size." So, basically, while the A9's 4 MiB L3 cache wouldn't have been much to add to the A9X, it wouldn't have helped much and a larger, more costly L3 cache would have been needed to help the A9X performance-wise. As the article points out, this is all speculation.
  - jacobolus 11y ago
    Their chart shows that the iPhone gets much lower latency for using up to 7 MB though http://images.anandtech.com/doci/9824/A9X_Latency2.png http://images.anandtech.com/doci/9824/A9X_Latency2.png There’s a huge latency gap between L2 cache and DRAM. Maybe not too many workloads need data chunks that fit into 7MB but can’t fit into 4MB.
  - CyberDildonics 11y ago
    A cache isn't a substitute for bandwidth, it helps with latency. More bandwidth doesn't impact that problem either way.
    - sliken 11y ago
      Actually it is. Every cache hit increases the available bandwidth for the CPU. So generally the larger the caches the less bandwidth and the less sensitive you are to memory latency.
      - CyberDildonics 11y ago
        A cache may free up bandwidth but it isn't a substitute for latency. Modern processors and vastly more latency bound then bandwidth bound, which is what cache most directly helps.
- paulmd 11y ago
  You mainly use cache for two reasons: speed and latency. Here, they eliminated the cache and used the die space for a really fast and wide bus to memory instead. So speed (total throughput) is covered. And from the article, it shows that the latency is actually better except within a narrow range of fetch sizes (~3000 to 8000 KiB). I suspect the better latency at the high end is because you no longer have to work through the caching logic (which does little in such large transfers anyway). And I guess their bet is that you're not going to make many transfers that happen to fall into that window. That seems like a good bet, my intuition is that there's a "bathtub" style distribution here where accesses are either reasonably random (and thus don't involve transferring more than ~3 MiB at a time) or they're straight-through and involve very large sequential transfers (greater than 7 MiB). Like the article notes, it seems to be an interesting decision. They speculate that the L3 cache on the A9 was actually a decision made for reasons of power efficiency (it takes less power to hit cache than to hit memory) and that wasn't a concern here, whereas they would have needed even more cache for the extra GPU cores, and die size was getting to be a concern. But they also note that the L3 cache didn't consume much space anyway, which is the weird thing to me.
- sliken 11y ago
  Keep in mind that the levels of caches increase the latency of a cache miss. So basically you want enough cache for the cachable bits and as little cache as possible for the non cachable bits. So higher bandwidth and many uses cases that involve streaming (like say 4k video or writing triangles from the GPU to the frame buffer) it makes sense to get rid of the L3 and upgrade the bandwidth to 50GB/sec. Hopefully Intel will match the bandwidth with their next generation. So far Intel's taken the easy route and added an off chip, but inside package cache for their highest end GPUs (Iris Pro) but left the memory bandwidth the same (128 bit @ DDR4 speeds around 25GB/sec.)
BooneJS 11y ago
I wonder if this was mostly a play for power savings? SRAM leakage power can be pretty high (relative to the total power budget) if it can't be powered down during sleep-like modes.
porsupah 11y ago
How would one, practically, go about modeling the impact of L3 impact in a SoC design, assuming you were intending to produce sonething like the A9X? (I speak as one who loved sim_g4 as a modeling tool for verifying compiler efficiency, or rather, occasional lack thereof. So easy to see when a cache miss would occur, or just execution unit contention)

30 points by lelf 11y ago

The complete lack of an L3 cache is fascinating if true. Not being an expert on chip design, I had thought the "number of cache levels" trend was monotonically increasing, especially since some of Intel's latest chips now have L4 caches, but now this. Can someone explain how they can get away with this? Is it enough to just have a really fast and wide bus, did they possibly need to make changes on the OS level, or is it just more acceptable on a device where you'll only be running one or two applications and very few background daemons/services?

stygiansonic 11y ago
The article had this to say: "One explanation may be that Apple deemed the L3 cache no longer necessary with the A9X’s 128-bit LPDDR4 memory bus; that 51.2GB/sec of bandwidth meant that they no longer needed the cache to avoid GPU stalls." And: "Our own Andrei Frumusanu suspects that it may be a power matter, and that Apple was using the L3 cache to save on power-expensive memory operations on the A9. With A9X however, it’s a tablet SoC that doesn’t face the same power restrictions, and as a result doesn’t need a power-saving cache. This would be coupled with the fact that with double the GPU cores, there would be a lot more pressure on just a 4MB cache versus the pressure created by A9, which in turn may drive the need for a larger cache and ultimately an even larger die size." So, basically, while the A9's 4 MiB L3 cache wouldn't have been much to add to the A9X, it wouldn't have helped much and a larger, more costly L3 cache would have been needed to help the A9X performance-wise. As the article points out, this is all speculation.
- jacobolus 11y ago
  Their chart shows that the iPhone gets much lower latency for using up to 7 MB though http://images.anandtech.com/doci/9824/A9X_Latency2.png http://images.anandtech.com/doci/9824/A9X_Latency2.png There’s a huge latency gap between L2 cache and DRAM. Maybe not too many workloads need data chunks that fit into 7MB but can’t fit into 4MB.
- CyberDildonics 11y ago
  A cache isn't a substitute for bandwidth, it helps with latency. More bandwidth doesn't impact that problem either way.
  - sliken 11y ago
    Actually it is. Every cache hit increases the available bandwidth for the CPU. So generally the larger the caches the less bandwidth and the less sensitive you are to memory latency.
    - CyberDildonics 11y ago
      A cache may free up bandwidth but it isn't a substitute for latency. Modern processors and vastly more latency bound then bandwidth bound, which is what cache most directly helps.
paulmd 11y ago
You mainly use cache for two reasons: speed and latency. Here, they eliminated the cache and used the die space for a really fast and wide bus to memory instead. So speed (total throughput) is covered. And from the article, it shows that the latency is actually better except within a narrow range of fetch sizes (~3000 to 8000 KiB). I suspect the better latency at the high end is because you no longer have to work through the caching logic (which does little in such large transfers anyway). And I guess their bet is that you're not going to make many transfers that happen to fall into that window. That seems like a good bet, my intuition is that there's a "bathtub" style distribution here where accesses are either reasonably random (and thus don't involve transferring more than ~3 MiB at a time) or they're straight-through and involve very large sequential transfers (greater than 7 MiB). Like the article notes, it seems to be an interesting decision. They speculate that the L3 cache on the A9 was actually a decision made for reasons of power efficiency (it takes less power to hit cache than to hit memory) and that wasn't a concern here, whereas they would have needed even more cache for the extra GPU cores, and die size was getting to be a concern. But they also note that the L3 cache didn't consume much space anyway, which is the weird thing to me.
sliken 11y ago
Keep in mind that the levels of caches increase the latency of a cache miss. So basically you want enough cache for the cachable bits and as little cache as possible for the non cachable bits. So higher bandwidth and many uses cases that involve streaming (like say 4k video or writing triangles from the GPU to the frame buffer) it makes sense to get rid of the L3 and upgrade the bandwidth to 50GB/sec. Hopefully Intel will match the bandwidth with their next generation. So far Intel's taken the easy route and added an off chip, but inside package cache for their highest end GPUs (Iris Pro) but left the memory bandwidth the same (128 bit @ DDR4 speeds around 25GB/sec.)

BooneJS 11y ago

I wonder if this was mostly a play for power savings? SRAM leakage power can be pretty high (relative to the total power budget) if it can't be powered down during sleep-like modes.

porsupah 11y ago

How would one, practically, go about modeling the impact of L3 impact in a SoC design, assuming you were intending to produce sonething like the A9X? (I speak as one who loved sim_g4 as a modeling tool for verifying compiler efficiency, or rather, occasional lack thereof. So easy to see when a cache miss would occur, or just execution unit contention)