4 ms·

> "if concerned with performance why are you running code as a shell script" Rewrite grep in python before running - got it. :P

• hnlmorg 4y ago
  That's not what I suggested. I was saying if you're writing performance sensitive code with a hot loop calling `egrep` then a smarter approach might be to use a language better tuned for performance which supports regex libraries. Shell scripts have their place and can out perform most languages if you're writing simple logic that can run across large datasets in parallel, such as cat very-large-file | grep "do not want" | sed -r 's/foo/bar/' > very-large-file-refactored (please excuse the useless use of `cat`, it's there to illustrate the direction of data flow) But in those types of scenarios the cost of $SHELL interpretation and fork() is massively outweighed by the savings of stream processing. So my point was: if you're writing a function which $SHELL interpretation and/or fork() create enough of a performance impact where you're looking to optimize how you exec `grep -E`, then maybe it's time to investigate whether $SHELL is the right language to write your function in.
  • jraph 4y ago
    > please excuse the useless use of `cat`, it's there to illustrate the direction of data flow Note that you can write < very-large-file Instead of cat very-large-file | To avoid a useless use of cat and keep the direction. Not that I care very much though. Using cat is less surprising to most people.
    • hnlmorg 4y ago
      > Using cat is less surprising to most people. Exactly :) STDIN redirection is a neat trick though so definitely worth highlighting. But it wouldn’t have helped with readability in my particular example.
  • chaxor 4y ago
    Perhaps I should have added a "/s", in addition to my ":p", but I appreciate the explanation here nonetheless