5 ms·

At this point, most performance gains stem from: 1. L1, L2, L3 cache sizes. Since these are implemented as S-RAM, they can drastically increase chip area, and

At this point, most performance gains stem from: 1. L1, L2, L3 cache sizes. Since these are implemented as S-RAM, they can drastically increase chip area, and hence cost. 2. ISA Acceleration of certain types of algorithms - like encryption, codecs and AI. There's absolutely nothing inherently lacking in the ARM ISA or architecture to impact #1, and ARM has consistently been adding instructions for #2, including SIMD support. Apple chips perform so well as Apple can afford more cache for the same $ cost, as they make their own chips. Android phone manufacturers need to pay more for Qualcomm to make their profits

baybal2 8y ago
You have to add more context... A12x advantage comes not only from gigantic caches, but in how efficient is the core in using them. Anybody well versed in microarchitecture design will tell that at some point increasing cache size will actually begin to slow you down. Both cache topology, address lookup logic, prefetch, and the dark magic like smart cache invalidation circuits matter a lot. The goal is to flush as few bits on cache miss as possible, find stored values in cache faster, and do prefetch efficiently. Only once you can do that, the cache size/core complexity tradeoff begins to work. Any normal CPU benchmark will be measuring a non accelerated workload. So, the measurements of A12x having a true lead thanks to wider pipeline, bigger caches, and smarter cache management are something very well expected.
- writepub 8y ago
  Agree. But the presumption is that eventually, cache management algorithms will reach parity with the competition/bleeding edge. Any and all parameters that can be tweaked will be. All top CPU arch companies hire from the same pool of PhDs/M.S' from top schools, after-all. Performance commoditization is truly upon us. ARM v.s. x86 is quite literally a battle of existing software binary support. Things like WASM and JIT will further obviate the need to worry about ISAs when making product design choices. As of 2019, you can't go wrong picking an ARM based design, given you picked a SOC with sufficient cache, etc. that your use case demands.
  - baybal2 8y ago
    Are you in the industry by any chance? On your position, I don't agree. Purpose built cores are there to stay. There is certainly no definite optimal cache configuration. And even inside ARM space, approaches vary dramatically (in part because cache algos is a patent minefield second only to wireless:) Samsung went on largely to non-determenistic algos for oop machinery, cache, and branch prediction; Qualcomm more or less went the Intel way by making fever, but faster execution units, and adapting cache for that; ARM always had size and cost in mind; and Apple did as aforementioned. Even very minimal changes to performance profile of execution side may turn things upside down for people engineering caches.
    - writepub 8y ago
      I think the stakes are so high, and funding so deep, that eventually cache management will reach parity, regardless of the choices made to get there. I'm not an insider to dissect algorithms academically, but if in the worst case one needs to license the competition's patent, that'll happen. Keep in mind that the cost of non-performancen is irrelevance in the market, and that's a non-option for the likes of Qualcomm, Samsung, etc.
      - baybal2 8y ago
        > one needs to license the competition's patent If it is competitor's, but much likely of a patent troll, whom are as many in this area as I said in wireless. Cache algos today are said to be designed very specifically to work around known traps, even if it meant going for suboptimal solutions. Are you stationed in China by any chance?
        writepub 8y ago
        > Are you stationed in China by any chance? Why do you ask? Where are you stationed?
        baybal2 8y ago
blattimwind 8y ago
> 1. L1, L2, L3 cache sizes. Since these are implemented as S-RAM, they can drastically increase chip area, and hence cost. The larger the cache the higher the lookup latency becomes. That's why L1 and L2 rarely increase in size. Athlon XP: 64+64 kB L1I/D (ok, this one had bigger caches than the competition, 16/32kB was more usual in that time frame); 256 kB L2 Zen: 64+32 kB L1I/D; 512 kB L2 Pentium III Tualatin: 16+16 LkB 1I/D; 256-512 kB L2 Skylake: 32+32 kB L1I/D; 256 kB L2
- writepub 8y ago
  You mean there's an upper limit to performance benefits of increasing cache sizes. The more important comparison is between a desktop CPU and ARM mobile. You'll quickly notice performance parity when cache sizes are increased on the mobile CPUs. And also notice Qualcomm's 845 and earlier lagged Apple's CPUs in cache size. The performance gains certainly plateau, but mobile caches weren't even trying to approach this limit, till now - explaining a big part of the performance gap between desktop and mobile
  - blattimwind 8y ago
    > You mean there's an upper limit to performance benefits of increasing cache sizes. A cache is a latency-hiding device over a working set of unknown size. The latency of the cache depends on its size. So picking a cache size embodies a huge set of unknowns (the programs that will be run, their working set sizes, latency and size of main and swap memories etc.). In any case, for a given workload you could draw up a graph of cache size (and implied cache latency) vs throughput / latency and see that there is a sweet spot (larger cache does not substantially increase performance, but rather degrades it, due to increased cache latency). Furthermore, when you do a shrink, you could put the implied performance gains into making a bigger cache of the same (absolute) latency, or making a cache of the same size with lower latency; the latter might be implicitly required since you now probably have a faster core that wants a cache with lower absolute latency.

baybal2 8y ago

You have to add more context... A12x advantage comes not only from gigantic caches, but in how efficient is the core in using them. Anybody well versed in microarchitecture design will tell that at some point increasing cache size will actually begin to slow you down. Both cache topology, address lookup logic, prefetch, and the dark magic like smart cache invalidation circuits matter a lot. The goal is to flush as few bits on cache miss as possible, find stored values in cache faster, and do prefetch efficiently. Only once you can do that, the cache size/core complexity tradeoff begins to work. Any normal CPU benchmark will be measuring a non accelerated workload. So, the measurements of A12x having a true lead thanks to wider pipeline, bigger caches, and smarter cache management are something very well expected.

writepub 8y ago
Agree. But the presumption is that eventually, cache management algorithms will reach parity with the competition/bleeding edge. Any and all parameters that can be tweaked will be. All top CPU arch companies hire from the same pool of PhDs/M.S' from top schools, after-all. Performance commoditization is truly upon us. ARM v.s. x86 is quite literally a battle of existing software binary support. Things like WASM and JIT will further obviate the need to worry about ISAs when making product design choices. As of 2019, you can't go wrong picking an ARM based design, given you picked a SOC with sufficient cache, etc. that your use case demands.
- baybal2 8y ago
  Are you in the industry by any chance? On your position, I don't agree. Purpose built cores are there to stay. There is certainly no definite optimal cache configuration. And even inside ARM space, approaches vary dramatically (in part because cache algos is a patent minefield second only to wireless:) Samsung went on largely to non-determenistic algos for oop machinery, cache, and branch prediction; Qualcomm more or less went the Intel way by making fever, but faster execution units, and adapting cache for that; ARM always had size and cost in mind; and Apple did as aforementioned. Even very minimal changes to performance profile of execution side may turn things upside down for people engineering caches.
  - writepub 8y ago
    I think the stakes are so high, and funding so deep, that eventually cache management will reach parity, regardless of the choices made to get there. I'm not an insider to dissect algorithms academically, but if in the worst case one needs to license the competition's patent, that'll happen. Keep in mind that the cost of non-performancen is irrelevance in the market, and that's a non-option for the likes of Qualcomm, Samsung, etc.
    - baybal2 8y ago
      > one needs to license the competition's patent If it is competitor's, but much likely of a patent troll, whom are as many in this area as I said in wireless. Cache algos today are said to be designed very specifically to work around known traps, even if it meant going for suboptimal solutions. Are you stationed in China by any chance?
      - writepub 8y ago
        > Are you stationed in China by any chance? Why do you ask? Where are you stationed?
        baybal2 8y ago

blattimwind 8y ago

> 1. L1, L2, L3 cache sizes. Since these are implemented as S-RAM, they can drastically increase chip area, and hence cost. The larger the cache the higher the lookup latency becomes. That's why L1 and L2 rarely increase in size. Athlon XP: 64+64 kB L1I/D (ok, this one had bigger caches than the competition, 16/32kB was more usual in that time frame); 256 kB L2 Zen: 64+32 kB L1I/D; 512 kB L2 Pentium III Tualatin: 16+16 LkB 1I/D; 256-512 kB L2 Skylake: 32+32 kB L1I/D; 256 kB L2

writepub 8y ago
You mean there's an upper limit to performance benefits of increasing cache sizes. The more important comparison is between a desktop CPU and ARM mobile. You'll quickly notice performance parity when cache sizes are increased on the mobile CPUs. And also notice Qualcomm's 845 and earlier lagged Apple's CPUs in cache size. The performance gains certainly plateau, but mobile caches weren't even trying to approach this limit, till now - explaining a big part of the performance gap between desktop and mobile
- blattimwind 8y ago
  > You mean there's an upper limit to performance benefits of increasing cache sizes. A cache is a latency-hiding device over a working set of unknown size. The latency of the cache depends on its size. So picking a cache size embodies a huge set of unknowns (the programs that will be run, their working set sizes, latency and size of main and swap memories etc.). In any case, for a given workload you could draw up a graph of cache size (and implied cache latency) vs throughput / latency and see that there is a sweet spot (larger cache does not substantially increase performance, but rather degrades it, due to increased cache latency). Furthermore, when you do a shrink, you could put the implied performance gains into making a bigger cache of the same (absolute) latency, or making a cache of the same size with lower latency; the latter might be implicitly required since you now probably have a faster core that wants a cache with lower absolute latency.