Don't you just take the current cpuid when you go to access the cache again? Then you mutex and access the per-cpu state. There is a tiny race window but 99.99

Don't you just take the current cpuid when you go to access the cache again? Then you mutex and access the per-cpu state. There is a tiny race window but 99.99% of the time you will be hitting the same cpu's cache as you just identified, and there will be ~zero contention. The main issue is thread preemption while you're holding a per-core cache mutex. Some other thread can't do meaningful work using the cache while the holder is sleeping.