3 ms·
> P.S. if you need to sort a huge file, just literally use the UNIX/Linux `sort` command. No, it does not load it all into RAM. It knows how to do chunked sorts
by rovr138 7y ago
> P.S. if you need to sort a huge file, just literally use the UNIX/Linux `sort` command. No, it does not load it all into RAM. It knows how to do chunked sorts, dump temp files into /tmp, and then merge them. Old school UNIX tools are smarter than you think.
This so much.
I’ve worked with many devs and admins that don’t understand the tools that they have at their disposal on their systems. They end up trying to reinvent the wheel and their solutions usually don’t consider all the edge cases
- saagarjha 7y agoNote that this does take a while; I let it go for about two hours before killing sort.
- rovr138 7y agotime gsort --parallel=2 -o $HOME/pwned-pass-sorted.txt pwned-passwords-sha1-ordered-by-count-v5.txt 2890.06 real 1400.86 user 165.54 sys 48mins `gsort` is GNU sort on coreutils (not the one included on macOS). This is on a Mac Mini 2011 (5,1) with the 2.3GHz i5. They really have a lot of things built into these tools :)