Cost/die size is part of the consideration especially for L3 but with L1 and L2 it is much more about latency than cost: bigger caches will have higher latency

Cost/die size is part of the consideration especially for L3 but with L1 and L2 it is much more about latency than cost: bigger caches will have higher latency than smaller ones. That's the entire reason for the multi-level cache hierarchy to exist in the first place. Graviton2 doesn't even have a particularly large total amount of cache, the only part of the cache system that's bigger than both AMD and Intel is the L1D. L1I is the same as AMD, L2 is same as Intel, L3 is smaller than both (talking about current-gen server chips). That L1D can indeed make a big difference in certain workloads though. The L3 is actually pretty small for how many cores there are, ARM recommends 1-2MB/core for the N1 cores in the Graviton2 and it has 512KB/core. AMD has 4MB per core in Zen2 albeit with a slightly weird setup where the L3 is localized to 4 core clusters.