3 ms·
Counterpoint: Allowing explicit control over prefetching or providing an additional "engine" which brings in data to caches in a pattern you like is esp. benefi
by bayindirh 3y ago
Counterpoint: Allowing explicit control over prefetching or providing an additional "engine" which brings in data to caches in a pattern you like is esp. beneficial in real-time and latency sensitive applications.
I have listened a talk where developers used such subsystem in the given processor. If unused, they would spend 95% of their time window just to copy the data, however by requesting the data ahead of time via that engine, they only used 10% of their time window to get the data, and accomplished what they wanted in ~50% of the time window they have, leaving tons of time for further features and improvements.
If x86 had a such feature, I'd use that in my Ph.D. to request the matrix data I'm accessing ahead of time, because the pattern I use is not linear but well defined. Now, if I want to accelerate that code further, I need to reorder my matrices to make the prefetcher happy, and refactor the whole codebase from top to bottom.
- theamk 3y agogeneric x86 does not, but SSE extension (present on all modern CPUs) does have it! and you naturally can use C to call it via intrinsics, because C is low level after all... https://stackoverflow.com/questions/48994494/how-to-properly-use-prefetch-instructions https://stackoverflow.com/questions/48994494/how-to-properly...
- JonChesterfield 3y agoYou may like the x86 instruction called 'prefetch'.
- monocasa 3y agoPREFETCHh: Prefetch Data Into Caches https://c9x.me/x86/html/file_module_x86_id_252.html https://c9x.me/x86/html/file_module_x86_id_252.html
- fanf2 3y agox86 does have a prefetch instruction, and it’s available as a compiler intrinsic in gcc and clang. But it’s really difficult to get a performance improvement from explicit prefetching on modern big CPUs because their dynamic prefetchers are very clever. Your data access pattern needs to be something the prefetcher does not know how to match, and you need to be able to prefetch addresses before the speculator would get to them by itself. Unlikely for matrices unless they have a very weird shape.
- bee_rider 3y agoIs the pattern linear for some stretches or something like that? I’m wondering if reordering your matrices is the right strategy anyway—you want to load a whole cache line at a time I guess, being able to program the prefetcher wouldn’t get you that, right?