4 ms·

Sleep sort is meant as a joke. Obviously, everybody understands there exists no O(1) GP sorting algorithm.

falcor84 5y ago
Assuming GP means General Purpose, yes, everyone who took an intro to algorithms should know that no sorting algorithm that relies on comparisons can do better than O(n*log(n)). But non-comparison algorithms [0] like radix sort [1] that rely on simplifying assumptions are not a joke and can be used in practice in many real-world cases. [0] https://en.wikipedia.org/wiki/Sorting_algorithm#Non-comparison_sorts https://en.wikipedia.org/wiki/Sorting_algorithm#Non-comparis... [1] https://en.wikipedia.org/wiki/Radix_sort https://en.wikipedia.org/wiki/Radix_sort
- lmilcin 5y ago
  That is why I specified "General Purpose". The moment you know something more about the data there might exist an opportunity to exploit that information to improve the algorithm. Which is an interesting topic in itself. Many times I have met people who said with absolute authority "no, it is not possible to implement this" only to be confronted with a counterexample. And that is because real life is rarely about spherical cows in vacuum. I have once implemented a transactional, log-based database for credit card terminal (20MHz ARM, 2MB total unified memory). The read (get value for key) operation was O(n^3) which the reviewers decided is "ABSOLUTELY FUCKING UNACCEPTABLE" (their words). The fun started when I asked them to suggest better implementation. They could not come up with anything even remotely as fast. And the basic reason was that n was guaranteed to be small as there wasn't even much space on the device for any more data. Other than that it exploited fantastic read ahead capability of the device plus it was so simple (basically couple nested loops) that CPU cache could be used very effectively. I remember their O(nlog(n)) was never faster and actually about 50 times slower in most operations.
  - bartvk 5y ago
    Amazing story, thanks for sharing!
  - _ea1k 5y ago
    I'm consistently amazed at how willing people are to make assumptions about optimizations with problems that don't fit their assumptions. Nice job.
    - lmilcin 5y ago
      I think main reason is our flawed education. The tasks you are being given at school are usually constructed so that they fit the idealized knowledge you just learned. What it teaches people is to short cut the critical path of reasoning which is to first make sure you actually understand the problem and then the problem actually fits the assumptions. People are not taught that even a small divergence from assumptions can have dramatic difference on the actual solution or applicability of what they think is a solution. They are also not taught the actual assumptions or spot how they look. They talk about CS algorithms as if they worked on idealized computers, but then run them on real ones. At school this is not taught because when it seems the problem fits the material it almost always is. People are actually rewarded for jumping to conclusions as this helps them go through material faster. If you don't learn anything about the world you bring your learning with you and try to apply it to real world and that's how a lot of developers operate. Especially in tech, you can usually look up solution to any problem easily. But for some reason this is the part everybody emphasizes rather than recognize knowing when to apply the solution is way more valuable. I also think it is part of the wisdom behind "premature optimization is root of all evil".
      - _tom_ 5y ago
        What is wrong with Leetcode and similar, in a nutshell.
  - IgorPartola 5y ago
    I once watched someone implement a heap in PHP in order to sort 4 domain names in order of priority and be able to resort them. The idea was I guess that at some point there may be 5 or even 6 of these and then we would really need the heap…
- Kranar 5y ago
  No general purpose sorting algorithm period can do better than O(n * log(n)), regardless of comparison or otherwise. For a radix sort with a complexity of O(n * w) where w is the key length, w converges to log(n) in the general case since the number of bits needed to represent n values grows logarithmically with n, hence you arrive back to O(n * log(n)). The O(n * log(n)) is an information theoretic result that applies even to radix sort, to be more precise the optimal sorting algorithm is Θ(n!). If you fix w to a constant, then certainly one can claim that radix sort is O(n), but then so are comparison sorts as well. That's not to say that radix sort isn't useful in practice, just because two sorting algorithms have the same asymptotic bounds doesn't mean they are equal, but it is to say that its performance is a matter of evaluating the constants rather than how it scales.
  - cyber_kinetist 5y ago
    Maybe it has more to do with the actual distribution/form of the data? From my understanding radix sort can be incredibly fast when the numbers have fixed digits and evenly distributed in a predefined range (like phone numbers) but will be ineffective for floating point numbers or uneven distributions with extreme outliers.
  - cb321 5y ago
    > since the number of bits needed to represent n values grows logarithmically with n What you say only applies to sorting n unique/distinct keys. With no prohibition on duplicate keys, n & w vary quite independently. In this more general setting, radix sorting is indeed "really" O(nw) and w could be like 8 bits for some giant data set (EDIT and order is absolutely a useful property even with dups since there can be satellite data with the keys - also possibly bounded!). This is a common talking past each other problem in this area. To your credit you made this assumption explicit. (EDIT: "General purpose" may have some semantic unclarity, I suppose.) In terms of relevance, everyone has their own, but my personal experience is that most (but not all) real world data allow both duplicate keys and bounding key width. In particular, the largest n data sets have been the least likely to have a prohibition on duplicate keys, and also the most likely to have small w (with a few exceptions). The random access pattern of memory writes in the passes of a radix sort may (or may not) be a problem, and constant factors matter, of course (as you also say). So tying w to n was has always seems a very special and somewhat artificial case to me. I also have never heard of any practical O(n) comparison sort for fixed w before. Can you perhaps give a reference to some such algorithm? Thanks!
air7 5y ago
Actually, for any specific size of N, every algorithm is O(1)...

Assuming GP means General Purpose, yes, everyone who took an intro to algorithms should know that no sorting algorithm that relies on comparisons can do better than O(n*log(n)). But non-comparison algorithms [0] like radix sort [1] that rely on simplifying assumptions are not a joke and can be used in practice in many real-world cases. [0] https://en.wikipedia.org/wiki/Sorting_algorithm#Non-comparison_sorts https://en.wikipedia.org/wiki/Sorting_algorithm#Non-comparis... [1] https://en.wikipedia.org/wiki/Radix_sort https://en.wikipedia.org/wiki/Radix_sort

lmilcin 5y ago
That is why I specified "General Purpose". The moment you know something more about the data there might exist an opportunity to exploit that information to improve the algorithm. Which is an interesting topic in itself. Many times I have met people who said with absolute authority "no, it is not possible to implement this" only to be confronted with a counterexample. And that is because real life is rarely about spherical cows in vacuum. I have once implemented a transactional, log-based database for credit card terminal (20MHz ARM, 2MB total unified memory). The read (get value for key) operation was O(n^3) which the reviewers decided is "ABSOLUTELY FUCKING UNACCEPTABLE" (their words). The fun started when I asked them to suggest better implementation. They could not come up with anything even remotely as fast. And the basic reason was that n was guaranteed to be small as there wasn't even much space on the device for any more data. Other than that it exploited fantastic read ahead capability of the device plus it was so simple (basically couple nested loops) that CPU cache could be used very effectively. I remember their O(nlog(n)) was never faster and actually about 50 times slower in most operations.
- bartvk 5y ago
  Amazing story, thanks for sharing!
- _ea1k 5y ago
  I'm consistently amazed at how willing people are to make assumptions about optimizations with problems that don't fit their assumptions. Nice job.
  - lmilcin 5y ago
    I think main reason is our flawed education. The tasks you are being given at school are usually constructed so that they fit the idealized knowledge you just learned. What it teaches people is to short cut the critical path of reasoning which is to first make sure you actually understand the problem and then the problem actually fits the assumptions. People are not taught that even a small divergence from assumptions can have dramatic difference on the actual solution or applicability of what they think is a solution. They are also not taught the actual assumptions or spot how they look. They talk about CS algorithms as if they worked on idealized computers, but then run them on real ones. At school this is not taught because when it seems the problem fits the material it almost always is. People are actually rewarded for jumping to conclusions as this helps them go through material faster. If you don't learn anything about the world you bring your learning with you and try to apply it to real world and that's how a lot of developers operate. Especially in tech, you can usually look up solution to any problem easily. But for some reason this is the part everybody emphasizes rather than recognize knowing when to apply the solution is way more valuable. I also think it is part of the wisdom behind "premature optimization is root of all evil".
    - _tom_ 5y ago
      What is wrong with Leetcode and similar, in a nutshell.
- IgorPartola 5y ago
  I once watched someone implement a heap in PHP in order to sort 4 domain names in order of priority and be able to resort them. The idea was I guess that at some point there may be 5 or even 6 of these and then we would really need the heap…
Kranar 5y ago
No general purpose sorting algorithm period can do better than O(n * log(n)), regardless of comparison or otherwise. For a radix sort with a complexity of O(n * w) where w is the key length, w converges to log(n) in the general case since the number of bits needed to represent n values grows logarithmically with n, hence you arrive back to O(n * log(n)). The O(n * log(n)) is an information theoretic result that applies even to radix sort, to be more precise the optimal sorting algorithm is Θ(n!). If you fix w to a constant, then certainly one can claim that radix sort is O(n), but then so are comparison sorts as well. That's not to say that radix sort isn't useful in practice, just because two sorting algorithms have the same asymptotic bounds doesn't mean they are equal, but it is to say that its performance is a matter of evaluating the constants rather than how it scales.
- cyber_kinetist 5y ago
  Maybe it has more to do with the actual distribution/form of the data? From my understanding radix sort can be incredibly fast when the numbers have fixed digits and evenly distributed in a predefined range (like phone numbers) but will be ineffective for floating point numbers or uneven distributions with extreme outliers.
- cb321 5y ago
  > since the number of bits needed to represent n values grows logarithmically with n What you say only applies to sorting n unique/distinct keys. With no prohibition on duplicate keys, n & w vary quite independently. In this more general setting, radix sorting is indeed "really" O(nw) and w could be like 8 bits for some giant data set (EDIT and order is absolutely a useful property even with dups since there can be satellite data with the keys - also possibly bounded!). This is a common talking past each other problem in this area. To your credit you made this assumption explicit. (EDIT: "General purpose" may have some semantic unclarity, I suppose.) In terms of relevance, everyone has their own, but my personal experience is that most (but not all) real world data allow both duplicate keys and bounding key width. In particular, the largest n data sets have been the least likely to have a prohibition on duplicate keys, and also the most likely to have small w (with a few exceptions). The random access pattern of memory writes in the passes of a radix sort may (or may not) be a problem, and constant factors matter, of course (as you also say). So tying w to n was has always seems a very special and somewhat artificial case to me. I also have never heard of any practical O(n) comparison sort for fixed w before. Can you perhaps give a reference to some such algorithm? Thanks!

air7 5y ago

Actually, for any specific size of N, every algorithm is O(1)...