Very cool, though not something a decent unixadmin can't do :) Note before you think this is "old crap" this approach scales to multi-core and multi-processor

Very cool, though not something a decent unixadmin can't do :) Note before you think this is "old crap" this approach scales to multi-core and multi-processor and can handle quite large sets. For really large sets you might need to play a bit with sort flags and perhaps do those steps it in two phases. Hardcore unixadmins often freak out developers when their month long projects on handling some files in production could be replaced with some scripting and in fact run it orders of magnitude faster. Multiprocessor means multiple L1 and L2 caches, less I/O blocking, etc. :)