4 ms·

Yeah perl was nice when we were allowed to use that. Why doesn't python have something like named pipes? f = open("ls|", "r) f.read() f.close()

• masklinn 7y ago
  I'm not sure what that's supposed to do, but subprocess provides these facilities, although definitely in a more verbose way. f = Popen('ls', stdout=PIPE).stdout f.read() f.close() alternatively given this exact behaviour: run('ls', stdout=PIPE).stdout
  • kamaal 7y ago
    Doesn't scale well when you have to stitch more than two commands with pipes, which is a very common use case for Unix CLI utilities.
    • masklinn 7y ago
      Don't see why it "doesn't scale well", the overhead is roughly constant: use the stdout of one program as the input of the next.
      • kamaal 7y ago
        That's not as easy as cat something | grep "this" | cut -f 1 | sed -e 's/.../.../' You end up writing too much code, it's very verbose. Some times symbols are what you want. In fact the biggest progress in the growth of Math happened when they tossed out doing math with words and bought in symbols.
        • Too 7y ago
          You forgot setting -e and -o pipefail. In case "something" is missing or can't be read for some other reason, your script will just continue happily without warning. But oh, if you do set -o pipefail the grep will stop the whole pipeline when none of the lines matches "this". So you have to keep fiddling with ${PIPESTATUS[0]}. And none of pipefail or PIPESTATUS are really portable. Not so simple after all.
        • masklinn 7y ago
          > That's not as easy as > cat something | grep "this" | cut -f 1 | sed -e 's/.../.../' Literally none of this is actually useful if you're already in Python: ( line.split('\t')[0].replace(…, …) for line in open('something') if 'this' in line ) > You end up writing too much code I can believe that if you're calling to external processes to perform operations which are pretty much trivial in the language.
          • kamaal 7y ago
            Of course, if you plan to use one $language alone, you can do anything in that $language. This question is specific to pipes.
            • masklinn 7y ago
              > Of course, if you plan to use one $language alone, you can do anything in that $language. That's missing what I'm noting though, which is that you don't need pipes anywhere near a shell script if you can simply do more of your work in-language. Case in point being that the entire pipeline you cited as an issue has no reason to exist outside of a shell or shell script.
              • kamaal 7y ago
                That's because regexes are an another pain in Python, You have to write a lot of boiler plate exception handling code. Plus the same boiler plate code for file operations. It doesn't feel like the language was designed for these tasks.
    • laurieodgers 7y ago
      I found this problem as well when doing a direct port including pipes from bash to python. It was the first iteration into python and I was lacking time to do a proper port into python native. The python code to do piping ends up as longwinded and the plumbing of pipes ends up a massive headache so I wrote tidycmd to overcome that issue https://github.com/laurieodgers/tidycmd https://github.com/laurieodgers/tidycmd
  • harry8 7y ago
    yeah sure subprocess.check_output(). If you ever dealt with much perl you know how much more pleasant and easy launching processes was in that language - like shell. Python is great, I'm not on the "bash python" wagon. Launching processes, getting the output, munging it and shoving it at another process is more fiddly and less natural in python. If you want to use that as evidence that I'm a garbage developer go ahead, I realise I'm not writing a carefully researched paper here with examples. In Perl you literally took the part of your shell pipeline, quoted it and opened it like any other file. ./output_generator | wc -l becomes open(SRC, "output_generator|"); open(WC, "|wc -l); And you do whatever you want with those filehandles. It's been ages since I wrote any perl. I miss it.
    • ThePadawan 7y ago
      You might like the python "sh" library [0]. It's not perfect, but it sure does remove the visual noise around subprocess. [0] https://amoffat.github.io/sh/ https://amoffat.github.io/sh/
    • masklinn 7y ago
      > yeah sure subprocess.check_output() It's deprecated and does something quite different. > If you ever dealt with much perl you know how much more pleasant and easy launching processes was in that language - like shell. I'm sure it is, but you're missing my point. > If you want to use that as evidence that I'm a garbage developer go ahead Well that escalated quickly. > open(SRC, "output_generator|"); > open(WC, "|wc -l); > > And you do whatever you want with those filehandles. Again python does roughly the same, just with more overhead: a trailing pipe is an "stdout=PIPE", an input is a "input=<whatever>", and you access / forward stdout explicitly: src = Popen('output_generator', stdout=PIPE) wc = Popen(['wc', '-l'], input=src.stdout) And as the sibling notes, for shell replacements you can use the sh library to lower the syntactic overhead of popen.
• deleted 7y ago
  [deleted]