3 ms·
Nice idea. If anyone is interested how the hell that works despite CPUs expose no instructions to handle cache misses in explicit way, here’s a quote from the l
by Const-me 6y ago
Nice idea. If anyone is interested how the hell that works despite CPUs expose no instructions to handle cache misses in explicit way, here’s a quote from the linked article.
To access data (e.g., a tree node) in request t1 which may incur a cache miss, the thread issues a prefetch and switches to another request t2, and repeats this process. While the data needed by t1 is being fetched from memory to CPU cache, the worker thread handles t2, which may further cause the thread to issue prefetch and switch to another request. By the time the worker thread switches back to t1, the hope is that the needed data is (already and still) cache-resident. The thread then picks up at where it left for t1, dereferences the pointer to the prefetched data and continues executing t1 until the next possible cache miss upon which a prefetch will be issued. It is important that the switching mechanism and representation of requests are cheap and lightweight enough to achieve a net gain.
- brooksbp 6y agoEven if you know which accesses have a high probability of missing in core or chip cache, it is tricky to write portable software that takes advantage of (switches/branches to other work) the time it takes between prefetch and data available in L1. It is not just architecture specific, or microarchitecture specific, it's chip specific.
- Const-me 6y agoNot sure I understand your comment. Many things may go wrong, interference from another hardware thread running on the same core, context switches, cache associativity shenanigans, and more. But still, if you prefetch, switch a co-routine so that other one is spending their 100-500 CPU cycles doing something else, then switch back — I would expect statistically significant performance win, at the cost of code complexity.
- lainga 6y agoI wonder if such explicit cache manipulation instructions will ever make their way into an x86_64 extension.
- Const-me 6y agoI know 2 related instructions, prefetch used by the OP, and clflush to flush a line. AFAIK there’s no way to query cache state for an address. With these recent spectre/meltdown/etc, I wouldn’t expect to see an instruction to query cache state for an address, at least in a foreseeable future. An easy implementation probably gonna be a side channel attack vector.
- PoignardAzur 6y agoAfaik the clflush instruction isn't really meant for optimization. It evinces data from all cache levels (so even from L3), which means it's only useful for data you know you won't ever need again. I think the CPU is good enough at flushing that data that additional hints won't improve performance much.
- PoignardAzur 6y agoThe instruction this is based on is prefetch; which is essentially the only relevant instruction when doing manual cache optimization on x86/x64. It's not a panacea though; naive usage of prefetch will actually decrease performance [0], because the hardware already does a lot of prefetching on its own and you're messing with that. From what little research I've done, prefetch mostly improves performance when accessing data in a hot loop, in a non-linear pattern (eg nextAddress isn't just prevAdress + constantValue), 3-4 loop iterations ahead of the execution. Eg doThings(myArrayOfPointers[i]); __builtin_prefetch(myArrayOfPointers[i + 4]); (though the above code might be undefined behavior if i + 4 is bigger than the array size, I'm not sure) [0]: https://lwn.net/Articles/444336/ https://lwn.net/Articles/444336/