3 ms·
Theoretical advances are constant (just check the literature), but two particular advances in the last decade are of practical importance: one for sort and one
by sanxiyn 3y ago
Theoretical advances are constant (just check the literature), but two particular advances in the last decade are of practical importance: one for sort and one for hash table. As in: check the implementation you are using, if it doesn't incorporate these advances you are leaving substantial performance (2x is not out of the question) on the table.
How Branch Mispredictions don't affect Quicksort (2016) https://arxiv.org/abs/1604.06697 https://arxiv.org/abs/1604.06697
Quicksort is still the algorithm of choice for general purpose internal unstable sort. In modern hardwares, quicksort spends a lot of time recovering from branch misprediction, because quicksort's comparison branches are inherently unpredictable. The problem is severe to the extent that with branchful quicksort, uneven split with more recursion is faster than even split, because it makes branches more predictable! Exact penalty is hardware microarchitecture specific, but one report is 20:80 split being optimal.
So... you should use branchless quicksort! The technique was pioneered by an implementation called BlockQuicksort, but pdqsort (for pattern-defeating quicksort) https://arxiv.org/abs/2106.05123 https://arxiv.org/abs/2106.05123 is also popular. pdqsort incorporates branchless technique and also includes other enhancements.
Designing a Fast, Efficient, Cache-friendly Hash Table, Step by Step (2017) https://abseil.io/blog/20180927-swisstables https://abseil.io/blog/20180927-swisstables
Hash table is the workhorse of data structures, and while improvements are possible for specific usage patterns, open addressing with linear probing is still the fastest implementation for general purpose. So called "Swiss Table" combines cache-friendly metadata layout with SIMD-enhanced probing. These two optimizations work well with each other and combination of two is particularly effective.
- hinkley 3y agoI recall years ago someone suggesting that a 1:2 split being faster and I'm wondering if they were seeing the same thing, and either didn't follow it through to conclusion or the goalposts keep moving in each generation of processors.
- sanxiyn 3y agoIt is plausible that they were seeing the same thing.
- anonymoushn 3y agoFor hit-heavy workloads, I have been unable to get Swiss tables to compete with tables that hit only ~1 cache line per query.
- _dain_ 3y ago>pdqsort (for pattern-defeating quicksort) I'm sad that it's not "pretty darn quicksort"
- sanxiyn 3y agoIt probably is that, just not officially. You know, officially, BFR stands for Big Falcon Rocket.
- RoyGBivCap 3y agoAnd the God(damn) Particle also comes to mind.
- xpe 3y ago> Theoretical advances are constant (just check the literature) * for some arbitrary definition of “constant” Some people throw the word “constant” around quite freely. Scientific and mathematical advances are notoriously hard to predict. “Constant” is the last word I would use. One simple model would be spaced in time according to a memoryless stochastic process while having “impact” scores sampled from an exponential distribution. Of course, one could include more realisticish dynamics to the model, such as network effects and clumping (e.g. from funding priorities and “ideas in the air”.) I feel like some areas of computer science tend to correlate with deep statistical appreciation more than others. Speaking of the topic of stochastic behavior in general, it feels to me like (a) randomized algorithms and (b) better statistical simulations have steadily been gaining traction in algorithms and data structures.