8 ms·

Often times it's easier to use binaries and manage their input/output through shell code. Having a good understanding of shell code will also allow you to easi

Often times it's easier to use binaries and manage their input/output through shell code. Having a good understanding of shell code will also allow you to easily write slick one-liners that will save you lots of time.

SOLAR_FIELDS 8y ago
For basic file system operations it’s a necessity. Write 10-20 line python script or type a one line grep/awk/sed piipeline to spit out the contents of some files?
- chrisfinazzo 8y ago
  There's a great talk by bwk where he complains about exactly this kind of behavior...before admitting that anything useful in a language will be abused in order to do silly things. https://www.youtube.com/watch?v=Sg4U4r_AgJU https://www.youtube.com/watch?v=Sg4U4r_AgJU I'll admit, I'm no fan of Python, but this has little to do with the language itself. I found Bash and Ruby first, so Python just seems like too much work to get right. Requisite xkcd: https://xkcd.com/1987/ https://xkcd.com/1987/
- jerf 8y ago
  It's a 10-20 line Python script if you write it without any support code. If you're writing any amount of Python code that does shell things, you should either write or get an existing library. Also, it really isn't 10-20 lines to spit out the contents of some files in Python: >>> filelist = ["tmp.pl", "tmp.go", "Tmp.hs"] >>> for f in filelist: ... print(open(f).read()) It's trivial to code golf that down to 1 line, but if we're writing in Python at any sort of scale, this is more realistic. Easy things are still generally easy. They just aren't optimized down to shell level in terms of keystroke count. They're often more optimized than bash in terms of conceptual complexity, though. For instance, if one of my files has a space in it, shell becomes a conceptual minefield, requiring you to understand the half-a-dozen ways it might process strings and how that interacts with filenames and arguments passed to commands, whereas the Python just keeps doing what you probably meant, because it knows what's a string and what's something else. Shell makes a lot of sacrifices for that concision, and there's a lot of "Well, this will probably be OK on anything I run it on...". I never put spaces in my file name, but use periods or underscores instead, precisely so I don't screw myself over with shell. I shouldn't have to do that. Personally I think the crossover point is in the high single digit number of bash lines. The only advantage bash still holds over Python at that point is large pipelines, and 80% of those can be replaced by Python functions. For the remainder, use a pipelining library.
  - bm1362 8y ago
    for i in ‘ls /tmp/foo’; do echo $i | grep bar | cut -d’-‘ -f3,4; done Is just so much easier to remember than Python’s OS library, right? I can intuitively string that together but can’t write python without looking up the docs and reading a paragraph about idiosyncrasies.
    - deleted 8y ago
      [deleted]
    - plorkyeran 8y ago
      This is going to come down to what you're most familiar with. I personally can never remember what the arguments to `cut` are due to rarely using it, while I'd be able to write the equivalent python from memory.
    - joshuamorton 8y ago
      Os.listdir isn't much harder to remember, and of course os.walk is actually probably easier for recursive filesystem actions.
    - jerf 8y ago
      I said Python wasn't going to be more concise. I said it has a variety of other useful properties. For instance, at least as I write this, you have "cut -d’-‘"; I assume you meant apostrophes (something getting too polite in a c&p, I assume) but the apostrophes are unnecessary, but you're so used to the compromises in shell I talked about you probably put those in there automatically. I'm not criticizing that; it's a reasonable accommodation to shell's quirks. But it is a quirk. Your code also breaks if the filenames have spaces: $ ls -1 bar-def-gh i-jkl-mno-p bar-def-ghi-jkl-mno-p $ for i in `ls`; do echo $i | grep bar | cut -d- -f3,4; done gh ghi-jkl The natural Python code you'd write to replace it will do what you expect and print the file with a space in it in the way you'd expect. You can fix this in bash via "for i in [asterisk]" (or, in this case, "for i in [asterisk]bar[asterisk]"), but, well, remember when I said bash requires you to understand the half-a-dozen ways strings and filenames interact...? In Python, there's just the one way that strings work. (HN is forcing me to say [asterisk]. AFAIK there's still no escaping for them.) You lucked out a bit here too; I don't think I can get echo to do anything very interesting based on file names. Had you interpolated that somewhere more interesting I'd give you some security issues too, if anybody not trusted can create files, which, again, the naive Python code would be much more robust to, because in Python, you have to explicitly ask to bring a string up to an execution context or a function parameter context, unlike shell's complicated rules for interleaving them. You pay a steep price for the convenience. Of course, this is a matter of scale. I use that sort of pipeline interactively every day. I just start shuddering when it goes in a file for a script, and shudder more if it goes into source control.

SOLAR_FIELDS 8y ago

For basic file system operations it’s a necessity. Write 10-20 line python script or type a one line grep/awk/sed piipeline to spit out the contents of some files?

chrisfinazzo 8y ago
There's a great talk by bwk where he complains about exactly this kind of behavior...before admitting that anything useful in a language will be abused in order to do silly things. https://www.youtube.com/watch?v=Sg4U4r_AgJU https://www.youtube.com/watch?v=Sg4U4r_AgJU I'll admit, I'm no fan of Python, but this has little to do with the language itself. I found Bash and Ruby first, so Python just seems like too much work to get right. Requisite xkcd: https://xkcd.com/1987/ https://xkcd.com/1987/
jerf 8y ago
It's a 10-20 line Python script if you write it without any support code. If you're writing any amount of Python code that does shell things, you should either write or get an existing library. Also, it really isn't 10-20 lines to spit out the contents of some files in Python: >>> filelist = ["tmp.pl", "tmp.go", "Tmp.hs"] >>> for f in filelist: ... print(open(f).read()) It's trivial to code golf that down to 1 line, but if we're writing in Python at any sort of scale, this is more realistic. Easy things are still generally easy. They just aren't optimized down to shell level in terms of keystroke count. They're often more optimized than bash in terms of conceptual complexity, though. For instance, if one of my files has a space in it, shell becomes a conceptual minefield, requiring you to understand the half-a-dozen ways it might process strings and how that interacts with filenames and arguments passed to commands, whereas the Python just keeps doing what you probably meant, because it knows what's a string and what's something else. Shell makes a lot of sacrifices for that concision, and there's a lot of "Well, this will probably be OK on anything I run it on...". I never put spaces in my file name, but use periods or underscores instead, precisely so I don't screw myself over with shell. I shouldn't have to do that. Personally I think the crossover point is in the high single digit number of bash lines. The only advantage bash still holds over Python at that point is large pipelines, and 80% of those can be replaced by Python functions. For the remainder, use a pipelining library.
- bm1362 8y ago
  for i in ‘ls /tmp/foo’; do echo $i | grep bar | cut -d’-‘ -f3,4; done Is just so much easier to remember than Python’s OS library, right? I can intuitively string that together but can’t write python without looking up the docs and reading a paragraph about idiosyncrasies.
  - deleted 8y ago
    [deleted]
  - plorkyeran 8y ago
    This is going to come down to what you're most familiar with. I personally can never remember what the arguments to `cut` are due to rarely using it, while I'd be able to write the equivalent python from memory.
  - joshuamorton 8y ago
    Os.listdir isn't much harder to remember, and of course os.walk is actually probably easier for recursive filesystem actions.
  - jerf 8y ago
    I said Python wasn't going to be more concise. I said it has a variety of other useful properties. For instance, at least as I write this, you have "cut -d’-‘"; I assume you meant apostrophes (something getting too polite in a c&p, I assume) but the apostrophes are unnecessary, but you're so used to the compromises in shell I talked about you probably put those in there automatically. I'm not criticizing that; it's a reasonable accommodation to shell's quirks. But it is a quirk. Your code also breaks if the filenames have spaces: $ ls -1 bar-def-gh i-jkl-mno-p bar-def-ghi-jkl-mno-p $ for i in `ls`; do echo $i | grep bar | cut -d- -f3,4; done gh ghi-jkl The natural Python code you'd write to replace it will do what you expect and print the file with a space in it in the way you'd expect. You can fix this in bash via "for i in [asterisk]" (or, in this case, "for i in [asterisk]bar[asterisk]"), but, well, remember when I said bash requires you to understand the half-a-dozen ways strings and filenames interact...? In Python, there's just the one way that strings work. (HN is forcing me to say [asterisk]. AFAIK there's still no escaping for them.) You lucked out a bit here too; I don't think I can get echo to do anything very interesting based on file names. Had you interpolated that somewhere more interesting I'd give you some security issues too, if anybody not trusted can create files, which, again, the naive Python code would be much more robust to, because in Python, you have to explicitly ask to bring a string up to an execution context or a function parameter context, unlike shell's complicated rules for interleaving them. You pay a steep price for the convenience. Of course, this is a matter of scale. I use that sort of pipeline interactively every day. I just start shuddering when it goes in a file for a script, and shudder more if it goes into source control.