5 ms·
In reality you never have a finely tuned load that uses all the resources all of the time. Processors spend a lot of time waiting for memory, so hyperthreading
by eloff 4y ago
In reality you never have a finely tuned load that uses all the resources all of the time. Processors spend a lot of time waiting for memory, so hyperthreading allows for better utilization of compute. It’s rare to have a workload that benefits from turning it off, and in those cases it’s usually because the hyperthread is hurting the cache hit rate enough to offset the gains.
- hinkley 4y agoBoth threads are waiting on the same memory pipeline. The actual performance has always lagged behind the theory
- eloff 4y agoBut not the same memory operations. Unless your code fully saturates the memory bandwidth, which is rare, you get some gains here.
- AdrianB1 4y agoIt is almost never about saturating the memory bandwidth, but waiting to load instructions and data from memory. That wait time counted in computer cycles is huge.
- eloff 4y agoYes, that’s precisely why hyperthreading is such a good deal.
- hinkley 4y ago“Such” a good deal is 2-30% in benchmarks (and they don’t say if that’s with cache leak protections turned on or off). In previous generations it was more like -20-20%. If one thread is having issues with L1 and L2 cache, splitting that evenly with a completely different workflow isn’t going to help. If cache contention weren’t a problem, and it was just a matter of jumping into previously unseen instructions and data (cold cache), you’d expect to see 50-300% numbers from hyperthreading, precisely because of how long the stalls are.
- kevin_thibedeau 4y agoIf one thread has to hit main memory but the other one can get what it needs from L1 cache they aren't competing for the same resources.
- MauranKilom 4y agoI recently checked whether an ugly hack I implemented years ago for 30% performance gains ("halve the default OpenMP thread count at start of computation and restore it afterwards" -> effectively, disable hyperthreading) was still necessary. It's now apparently a 3x performance gain... All because I'm saturating memory bandwidth. I don't know whether to be happy or sad...
- eloff 4y agoOuch, looks like they’re clobbering each others cache. Also, well done. It looks like you optimized the hell out of your code.