There are challenges with really big monolithic caches. IBM does something sort of like your idea in their Power and Telum chips, with different approaches. Pow

There are challenges with really big monolithic caches. IBM does something sort of like your idea in their Power and Telum chips, with different approaches. Power has a non-uniform cache within each die, Telum has a way to stitch together cache even across sockets (!). https://chipsandcheese.com/p/telum-ii-at-hot-chips-2024-mainframe-with-a-unique-caching-strategy https://chipsandcheese.com/p/telum-ii-at-hot-chips-2024-main... https://www.eecg.utoronto.ca/~moshovos/ACA07/projectsuggestions/ieee_micro04_nuca.pdf https://www.eecg.utoronto.ca/~moshovos/ACA07/projectsuggesti... (if you do ML things you might recognize Doug Burger's name on the authors line of the second one)