2 ms·
> my hands start shaking when a computer does not answer immediately I’m sorry to hear that. It sounds like it could be an anxiety issue somewhat unrelated to
by dahart 4y ago
> my hands start shaking when a computer does not answer immediately
I’m sorry to hear that. It sounds like it could be an anxiety issue somewhat unrelated to what we are talking about? Does it help to stick to very small files so you never have to wait? Does it help when you get progress updates in the interface?
I still don’t understand what you’re suggesting because the idea that you want to give arbitrarily sized inputs to a sort and have it come back instantly is unrealistic. You can only improve the overheads and bandwidth, you can never avoid waiting if you give larger inputs. No matter what solution you suggest, I can give you a file you’ll have to wait for. Speeding up the unix pipeline doesn’t appear to solve the problem you’re describing, that’s why I’m asking for more detail.
> it’s in the original post
What is? I’m not sure what you’re referring to. You only mean the imaginary idea to combine 3 processes into one? @yegle only appeared to ask for a shortcut for a common workflow, and not a more performant alternative. I’m asking you what your preferred practical implementation is. What can you do today to speed up this workflow? And how does that solve the problem of having to wait?
> the uniq can stream but the sorts can’t
Right, this is true regardless of what tool you use. You can eliminate the overheads of starting up 3 processes, but you can’t avoid the second sort, if you ask for a second sort.
Might be worth observing that the overheads of 3 processes get relatively smaller the larger the inputs are. If you’re sorting a gigabyte or terabyte of text, using a single process will be barely better, if at all.
Might also be worth looking at cold cache vs warm cache startup times for 3 processes. Does your 2/3 second change the second time?
PS GNU sort has options that can improve its bandwidth. Maybe try these? https://www.reddit.com/r/commandline/comments/a7hq5n/psa_improving_gnu_sort_speed/ https://www.reddit.com/r/commandline/comments/a7hq5n/psa_imp...
This is another reason to avoid writing your own tool, since sort may in reality have a lot of functionality that is not easy to duplicate, and may have ways to gain performance that is hard to beat.