5 ms·

That Python example is broken in all kinds of ways, and is overly verbose. I would go with something like the following, which does spark some joy in me: m

That Python example is broken in all kinds of ways, and is overly verbose. I would go with something like the following, which does spark some joy in me: matches = {l for l in open("beef.txt") if "lasagna" in l} print(*sorted(matches, reverse=True), sep="") On And on the same note, the shell command is weird too, and should probably be: grep "lasagna" beef.txt | sort -nur So the shell command is still definitely simpler for this case, but for anything more complex, I'd go with Python. EDIT: I put back the `n` flag, in case the lines start with numbers.

sigg3 4y ago
Sort has -u too.
- benj111 4y ago
  I wonder if that's a part of the issue. There's almost no fixed grammar. Especially the basic things, many programs do the thing as an option. Imagine if python functions /classes had their own add implementation. You aren't just looking for foo + bar. You've also got baz(foo, bar, add), biz(add, foo, bar), buz(foo, bar).add(). Some people will use one. Some another, they'll be subtle incompatilities, etc.
BoppreH 4y ago
That's much better, but like the original implementation it's still missing the -n flag for sort, which extracts numbers at the beginning of lines and sorts them numerically instead of lexicographically.
- falcor84 4y ago
  Oh, I always thought it only works on numbers, and am very pleasantly surprised to learn it works on numeric prefixes, thank you! EDIT: After playing around with the python approach, I arrived at this, which is definitely more verbose, but arguably, more maintainable: import re NUMERIC_PREFIX_RE = re.compile(r"^(\d+.?\d*)") def numeric_descending(line): if m := NUMERIC_PREFIX_RE.match(line): return -float(m.group()) return float("inf") matches = {l for l in open("beef.txt") if "lasagna" in l} print(*sorted(matches, key=numeric_descending), sep="")
  - pama 4y ago
    Not sure it is more maintainable than a shell solution. Look at this in two years from now thinking there might be a bug somewhere in there (say you want to include negatives and scientific notation). In the shell, a quick inspection suffices. In this case you have to deal with regular expressions within Python, their own little special language… and it may not spark joy.
    - falcor84 4y ago
      Well, it seems you nerd-sniped me a bit, so here's a new version which deals well with scientific notation (negatives were already supported) and is actually simpler, using a split and "it's better to ask for forgiveness..." instead of the regex: def numeric_descending(line): possible_number = line.split()[0] try: return (-float(possible_number), line) except ValueError: return (float("inf"), line) matches = {l for l in open("beef.txt") if "lasagna" in l} print(*sorted(matches, key=numeric_descending), sep="")
      - pama 4y ago
        Thanks, and apologies---I didn't mean to come off as negative. Yes, this code is cleaner and certainly maintainable a couple years down the road. It will work in a slightly different way than the shell script (the lack of space in "10oz" of lasagna vs "2.0e1 oz" of lasagna), however I'd consider this a feature rather than a bug, as it could help identify bad formatting in the input data. I've learned from experience that it is hard to replace certain simple space and memory efficient programs that have been tuned over the course of 40 or more years. If one wanted to print only a single instance of a line in a file, then python's set construct in your snippet is certainly sparking more joy compared to the awk equivalent of printing only the first instance of a line in a file: awk '!x[$0] {print; x[$0]=1}' Tuning python to use ordered dicts to keep the line order looks a little less pleasant (and might appear confusing to an uninitiated), but would still be rather neat. There are many instance of day-to-day data inspection tasks where bash, sort, shuf, sed, grep (or rg), and awk will solve a problem near optimally out of the box and can combine nicely in modest memory machines to tackle large data, when some python native code (or poorly designed python library) might accidentally cause a performance mess. Interactive shells are an awesome superpower that is hopefully going to stay with us for a while. I'd have personally been happy if some type of Lisp REPL had won over the shell fights, but we don't live in that world, so python + posix tools via a shell will have to work in practice for most people.
smcl 4y ago
Your Python version is def neater and more compact. However there's something really elegant about using the | operator to compose commands (or using |> in OCaml/F# or iirc $ in Haskell) that I can't quite articulate :)
- maest 4y ago
  > something really elegant about using the | operator For me, it's the fact that it nicely maps to high level steps I would use to manipulate the option, expressed quite tersely.
  - smcl 4y ago
    What I mean is that even if the steps would otherwise be identical, I can't really explain why I feel like structuring them like this: get_users |> filterActiveUsers |> sortByDateCreated |> getEmail or even this: get_users |> filterActiveUsers |> sortByDateCreated |> getEmail feels neater than: getEmail(sortByDateCreated(filterActiveUsers(get_users()))) My example is a bit stupid but I hope it's clear what I mean. Maybe it's just that as someone who only knows left-to-right languages, the flow of the data matches the order the steps appear? Actually I'm now curious if Arabic or Hebrew speakers feel the same way. Or maybe I'm overthinking this :)
    - cout 4y ago
      For me, the last example feels messy for two reasons. First, I have to worry about matching parens. That can feel mentally exhausting even for simple cases. Second, it's written all on one line, so it's harder to mentally separate all the steps. I usually break expressions like that apart into separate lines, but that means finding concise yet meaningful names for temporary variables.
      - smcl 4y ago
        Oh yeah I would definitely split them up into little intermediate steps. But then I’d get a bit sad that I don’t have the pipe operator :-)
crabbone 4y ago
I noticed the contrived Python example too, but yours will explode if the input is too big, so, it's not equivalent to Shell, which will buffer the input. Still, even with that in mind, Python variant could've been greatly simplified. To give the author the benefit of a doubt: they probably made such a contrived example on purpose to underscore their point, but it's hard to find small-scale examples which illustrate verbosity... On the other hand, I might be overthinking on behalf of the author. Doubt they explicitly set out to make a contrived example. They probably wrote what they wrote and were happy with the result because it helped them to make a compelling argument, and they didn't think to analyze their own illustrations too deeply to see if they might disagree with their conclusions.
- falcor84 4y ago
  >yours will explode if the input is too big, so, it's not equivalent to Shell, which will buffer the input. I agree that mine would explode on a massive file, but how would the shell version avoid this, given the need to sort? As far as I know, it sorts entirely in memory.
  - Karellen 4y ago
    When the input is large enough, GNU `sort` writes the input in chunks to multiple temporary files, sorts the individual files, and then merges the result for output. https://unix.stackexchange.com/questions/279096/scalability-of-sort-u-for-gigantic-files https://unix.stackexchange.com/questions/279096/scalability-...

sigg3 4y ago

Sort has -u too.

benj111 4y ago
I wonder if that's a part of the issue. There's almost no fixed grammar. Especially the basic things, many programs do the thing as an option. Imagine if python functions /classes had their own add implementation. You aren't just looking for foo + bar. You've also got baz(foo, bar, add), biz(add, foo, bar), buz(foo, bar).add(). Some people will use one. Some another, they'll be subtle incompatilities, etc.

BoppreH 4y ago

That's much better, but like the original implementation it's still missing the -n flag for sort, which extracts numbers at the beginning of lines and sorts them numerically instead of lexicographically.

falcor84 4y ago
Oh, I always thought it only works on numbers, and am very pleasantly surprised to learn it works on numeric prefixes, thank you! EDIT: After playing around with the python approach, I arrived at this, which is definitely more verbose, but arguably, more maintainable: import re NUMERIC_PREFIX_RE = re.compile(r"^(\d+.?\d*)") def numeric_descending(line): if m := NUMERIC_PREFIX_RE.match(line): return -float(m.group()) return float("inf") matches = {l for l in open("beef.txt") if "lasagna" in l} print(*sorted(matches, key=numeric_descending), sep="")
- pama 4y ago
  Not sure it is more maintainable than a shell solution. Look at this in two years from now thinking there might be a bug somewhere in there (say you want to include negatives and scientific notation). In the shell, a quick inspection suffices. In this case you have to deal with regular expressions within Python, their own little special language… and it may not spark joy.
  - falcor84 4y ago
    Well, it seems you nerd-sniped me a bit, so here's a new version which deals well with scientific notation (negatives were already supported) and is actually simpler, using a split and "it's better to ask for forgiveness..." instead of the regex: def numeric_descending(line): possible_number = line.split()[0] try: return (-float(possible_number), line) except ValueError: return (float("inf"), line) matches = {l for l in open("beef.txt") if "lasagna" in l} print(*sorted(matches, key=numeric_descending), sep="")
    - pama 4y ago
      Thanks, and apologies---I didn't mean to come off as negative. Yes, this code is cleaner and certainly maintainable a couple years down the road. It will work in a slightly different way than the shell script (the lack of space in "10oz" of lasagna vs "2.0e1 oz" of lasagna), however I'd consider this a feature rather than a bug, as it could help identify bad formatting in the input data. I've learned from experience that it is hard to replace certain simple space and memory efficient programs that have been tuned over the course of 40 or more years. If one wanted to print only a single instance of a line in a file, then python's set construct in your snippet is certainly sparking more joy compared to the awk equivalent of printing only the first instance of a line in a file: awk '!x[$0] {print; x[$0]=1}' Tuning python to use ordered dicts to keep the line order looks a little less pleasant (and might appear confusing to an uninitiated), but would still be rather neat. There are many instance of day-to-day data inspection tasks where bash, sort, shuf, sed, grep (or rg), and awk will solve a problem near optimally out of the box and can combine nicely in modest memory machines to tackle large data, when some python native code (or poorly designed python library) might accidentally cause a performance mess. Interactive shells are an awesome superpower that is hopefully going to stay with us for a while. I'd have personally been happy if some type of Lisp REPL had won over the shell fights, but we don't live in that world, so python + posix tools via a shell will have to work in practice for most people.

smcl 4y ago

Your Python version is def neater and more compact. However there's something really elegant about using the | operator to compose commands (or using |> in OCaml/F# or iirc $ in Haskell) that I can't quite articulate :)

maest 4y ago
> something really elegant about using the | operator For me, it's the fact that it nicely maps to high level steps I would use to manipulate the option, expressed quite tersely.
- smcl 4y ago
  What I mean is that even if the steps would otherwise be identical, I can't really explain why I feel like structuring them like this: get_users |> filterActiveUsers |> sortByDateCreated |> getEmail or even this: get_users |> filterActiveUsers |> sortByDateCreated |> getEmail feels neater than: getEmail(sortByDateCreated(filterActiveUsers(get_users()))) My example is a bit stupid but I hope it's clear what I mean. Maybe it's just that as someone who only knows left-to-right languages, the flow of the data matches the order the steps appear? Actually I'm now curious if Arabic or Hebrew speakers feel the same way. Or maybe I'm overthinking this :)
  - cout 4y ago
    For me, the last example feels messy for two reasons. First, I have to worry about matching parens. That can feel mentally exhausting even for simple cases. Second, it's written all on one line, so it's harder to mentally separate all the steps. I usually break expressions like that apart into separate lines, but that means finding concise yet meaningful names for temporary variables.
    - smcl 4y ago
      Oh yeah I would definitely split them up into little intermediate steps. But then I’d get a bit sad that I don’t have the pipe operator :-)

crabbone 4y ago

I noticed the contrived Python example too, but yours will explode if the input is too big, so, it's not equivalent to Shell, which will buffer the input. Still, even with that in mind, Python variant could've been greatly simplified. To give the author the benefit of a doubt: they probably made such a contrived example on purpose to underscore their point, but it's hard to find small-scale examples which illustrate verbosity... On the other hand, I might be overthinking on behalf of the author. Doubt they explicitly set out to make a contrived example. They probably wrote what they wrote and were happy with the result because it helped them to make a compelling argument, and they didn't think to analyze their own illustrations too deeply to see if they might disagree with their conclusions.

falcor84 4y ago
>yours will explode if the input is too big, so, it's not equivalent to Shell, which will buffer the input. I agree that mine would explode on a massive file, but how would the shell version avoid this, given the need to sort? As far as I know, it sorts entirely in memory.
- Karellen 4y ago
  When the input is large enough, GNU `sort` writes the input in chunks to multiple temporary files, sorts the individual files, and then merges the result for output. https://unix.stackexchange.com/questions/279096/scalability-of-sort-u-for-gigantic-files https://unix.stackexchange.com/questions/279096/scalability-...