3 ms·
> Am I wrong to avoid writing O(n^2) code if at all possible when it is fairly easy to use hash tables for a better time complexity Are you sure that std::unor
by dragontamer 6y ago
> Am I wrong to avoid writing O(n^2) code if at all possible when it is fairly easy to use hash tables for a better time complexity
Are you sure that std::unordered_map is faster than std::vector? Did you measure?
Every time you access an element in std::vector, you also access nearby ones (thanks to L1 cache, as well as CPU-prefetching of in-line data).
In contrast, your std::unordered_map or hash-table has almost no benefits to L1 cache. (It should be noted that linear-probing, despite being a O(N^2) version of hash-tables worst-case, is actually one of the better performers due to L1 cache + prefetching)
- zabzonk 6y agoAlso, creating a hash isn't free. And often ordering is required.
- dragontamer 6y ago> Also, creating a hash isn't free. Hmmm... I argue that the hash is nearly free actually. An unordered_map traversal is probably DDR4 latency bound. That's ~50-nanoseconds (200 clock ticks) per access. What's the CPU going to do in that time? Well, spending 10 to 20 clock ticks on a typical hash algorithm is fine. Then it will wait the other 180 clock ticks for RAM. If you got hyperthreading, maybe the CPU will go to another thread and do meaningful work while waiting for RAM... but... I think you get the gist. Even IF the hash were free, the CPU is waiting for RAM anyway. So you got plenty of time to make that hash worthwhile. Even an integer division/modulo operator (worst case ~80 clock ticks) can fit in there while waiting for RAM, with plenty of room to spare. I guess if everything was in L1 cache, the story is a bit different. A lot of "depends", depends on the data, the access frequency, etc. etc.
- krzat 6y agoWorrying about performance of small collections is premature optimization. Using maps or sets nowadays is mostly for clarity, as they are used to solve certain kind of problems.
- dragontamer 6y agoI agree with you. But what you're talking about is completely different from what I was responding to originally. If you need a set, use a set. But don't assume that its faster than a std::vector. Even then, std::vector has set-like operations through binary_search or std::make_heap in C++, so it really isn't that hard using a sorted (or make_heap'd) std::vector in practice. -------- Even if you don't plan on doing optimization work, its important to have a proper understanding of a modern CPU. The effects of L1 and prefetching are non-trivial, and make simple arrays and std::vectors extremely fast data structures, far faster than compared to 80s or 90s computers anyway. A lot of optimization advice from the past has become outdated because of the evolution of CPUs. So its important to bring up these changes in discussion, from time to time, to remind others to restudy computers. Things change.
- avasthe 6y ago> sorted std::vector But inserting into sorted std::vector is o(n log n) in worst case right? (As you have to binary search the position and move other elements). But a hash set with linear probing can give O(1) (amortized) access, while usually maintaining an invariant like total_size > 2*n. I don't think that would be such an impact on cache locality. Linear probing doesn't require linked lists. Of course this is given that you have a data structure in standard library. But at this point I think hash sets are pretty standard.
- dragontamer 6y agoYeah, linear probing helps hash-sets a lot on modern CPUs. Perhaps linear probing is a better example of how BigO analysis can go wrong on modern architectures. Inserting into a hashset with linear-probing is O(n) worst-case, while inserting into a linked list is always O(1) (best case, worst case, and average case). And yet, linear probing seems to work out best in practice (with a bit of rigging. The total_size > 2*n invariant is one, but so does Robin-hood hashing if you want to keep the table small) Linear probing vs Linked List implementations of hash-sets seems to be a more clear example of an O(1) vs O(n) anomaly, where the O(n) example is superior.
- zelly 6y agoTo be fair std::unordered_map is one of the slowest hash tables in any programming language.