5 ms·

for i in ‘ls /tmp/foo’; do echo $i | grep bar | cut -d’-‘ -f3,4; done Is just so much easier to remember than Python’s OS library, right? I can intuitively str

by bm1362 8y ago

for i in ‘ls /tmp/foo’; do echo $i | grep bar | cut -d’-‘ -f3,4; done Is just so much easier to remember than Python’s OS library, right? I can intuitively string that together but can’t write python without looking up the docs and reading a paragraph about idiosyncrasies.

deleted 8y ago
[deleted]
plorkyeran 8y ago
This is going to come down to what you're most familiar with. I personally can never remember what the arguments to `cut` are due to rarely using it, while I'd be able to write the equivalent python from memory.
joshuamorton 8y ago
Os.listdir isn't much harder to remember, and of course os.walk is actually probably easier for recursive filesystem actions.
jerf 8y ago
I said Python wasn't going to be more concise. I said it has a variety of other useful properties. For instance, at least as I write this, you have "cut -d’-‘"; I assume you meant apostrophes (something getting too polite in a c&p, I assume) but the apostrophes are unnecessary, but you're so used to the compromises in shell I talked about you probably put those in there automatically. I'm not criticizing that; it's a reasonable accommodation to shell's quirks. But it is a quirk. Your code also breaks if the filenames have spaces: $ ls -1 bar-def-gh i-jkl-mno-p bar-def-ghi-jkl-mno-p $ for i in `ls`; do echo $i | grep bar | cut -d- -f3,4; done gh ghi-jkl The natural Python code you'd write to replace it will do what you expect and print the file with a space in it in the way you'd expect. You can fix this in bash via "for i in [asterisk]" (or, in this case, "for i in [asterisk]bar[asterisk]"), but, well, remember when I said bash requires you to understand the half-a-dozen ways strings and filenames interact...? In Python, there's just the one way that strings work. (HN is forcing me to say [asterisk]. AFAIK there's still no escaping for them.) You lucked out a bit here too; I don't think I can get echo to do anything very interesting based on file names. Had you interpolated that somewhere more interesting I'd give you some security issues too, if anybody not trusted can create files, which, again, the naive Python code would be much more robust to, because in Python, you have to explicitly ask to bring a string up to an execution context or a function parameter context, unlike shell's complicated rules for interleaving them. You pay a steep price for the convenience. Of course, this is a matter of scale. I use that sort of pipeline interactively every day. I just start shuddering when it goes in a file for a script, and shudder more if it goes into source control.
- bm1362 8y ago
  Sorry I wrote that in a taxi from my phone, not a serious usecase but that’s beside the point I was trying to make- I personally have a hard time using python over bash. Thanks for pointing out the security aspect, I don’t typically think about that.
  - jerf 8y ago
    I figured; you clearly knew enough bash to know what the real backticks are. Those... frontticks?... aren't even on most keyboards. On the plus side, when I fixed the typographical issues, it did work. I'm pretty sure I could do this more-or-less first-time right in Python, too, but it clearly took me longer to get there.
- Too 8y ago
  It's hilarious in these discussions that every time someone comes to show how they can do the equivalent code in much less lines of bash there are always always always bugs of these sorts. And then the parent even specifically pointed out that files with spaces would be a minefield. Also complaining about looking up docs is just a matter of where you have your knowledge. I would have the opposite problem to understand the "idiosyncrasies" of what the -d flag into cut does whereas I know the python standard library by heart.
  - bm1362 8y ago
    You’re taking HN too seriously, it’s not a formal argument for nitpicking - it’s an opinion.
- Nimitz14 8y ago
  You seem to be imagining some sort of situation where bash is used to deal with input from customers (random users). You do not use bash for that. Filenames with spaces (lol) do not happen unless you do it yourself. Bash is useful for productivity when getting stuff done. For the work I do (where code never comes close to a customer and one-off tasks are common), if a candidate were to write a python script for the example line of bash I would be strongly biased towards rejecting them.
squeaky-clean 8y ago
That's just one personal anecdote though. I can do the Python example from memory but not the bash example.
Lio 8y ago
One nice thing about using a pipeline is that it will be multi-threaded by default. That is each process will have it's own thread(s) and distribute any work across multiple cores. Also with something like Gnu Parallel you can easily scale across multiple machines. Maybe there are more efficient approaches but a pipeline and gnu parallel is pretty easy to scale up without thinking much about it.

by bm1362 8y ago

deleted 8y ago

[deleted]

plorkyeran 8y ago

This is going to come down to what you're most familiar with. I personally can never remember what the arguments to `cut` are due to rarely using it, while I'd be able to write the equivalent python from memory.

joshuamorton 8y ago

Os.listdir isn't much harder to remember, and of course os.walk is actually probably easier for recursive filesystem actions.

jerf 8y ago

I said Python wasn't going to be more concise. I said it has a variety of other useful properties. For instance, at least as I write this, you have "cut -d’-‘"; I assume you meant apostrophes (something getting too polite in a c&p, I assume) but the apostrophes are unnecessary, but you're so used to the compromises in shell I talked about you probably put those in there automatically. I'm not criticizing that; it's a reasonable accommodation to shell's quirks. But it is a quirk. Your code also breaks if the filenames have spaces: $ ls -1 bar-def-gh i-jkl-mno-p bar-def-ghi-jkl-mno-p $ for i in `ls`; do echo $i | grep bar | cut -d- -f3,4; done gh ghi-jkl The natural Python code you'd write to replace it will do what you expect and print the file with a space in it in the way you'd expect. You can fix this in bash via "for i in [asterisk]" (or, in this case, "for i in [asterisk]bar[asterisk]"), but, well, remember when I said bash requires you to understand the half-a-dozen ways strings and filenames interact...? In Python, there's just the one way that strings work. (HN is forcing me to say [asterisk]. AFAIK there's still no escaping for them.) You lucked out a bit here too; I don't think I can get echo to do anything very interesting based on file names. Had you interpolated that somewhere more interesting I'd give you some security issues too, if anybody not trusted can create files, which, again, the naive Python code would be much more robust to, because in Python, you have to explicitly ask to bring a string up to an execution context or a function parameter context, unlike shell's complicated rules for interleaving them. You pay a steep price for the convenience. Of course, this is a matter of scale. I use that sort of pipeline interactively every day. I just start shuddering when it goes in a file for a script, and shudder more if it goes into source control.

bm1362 8y ago
Sorry I wrote that in a taxi from my phone, not a serious usecase but that’s beside the point I was trying to make- I personally have a hard time using python over bash. Thanks for pointing out the security aspect, I don’t typically think about that.
- jerf 8y ago
  I figured; you clearly knew enough bash to know what the real backticks are. Those... frontticks?... aren't even on most keyboards. On the plus side, when I fixed the typographical issues, it did work. I'm pretty sure I could do this more-or-less first-time right in Python, too, but it clearly took me longer to get there.
Too 8y ago
It's hilarious in these discussions that every time someone comes to show how they can do the equivalent code in much less lines of bash there are always always always bugs of these sorts. And then the parent even specifically pointed out that files with spaces would be a minefield. Also complaining about looking up docs is just a matter of where you have your knowledge. I would have the opposite problem to understand the "idiosyncrasies" of what the -d flag into cut does whereas I know the python standard library by heart.
- bm1362 8y ago
  You’re taking HN too seriously, it’s not a formal argument for nitpicking - it’s an opinion.
Nimitz14 8y ago
You seem to be imagining some sort of situation where bash is used to deal with input from customers (random users). You do not use bash for that. Filenames with spaces (lol) do not happen unless you do it yourself. Bash is useful for productivity when getting stuff done. For the work I do (where code never comes close to a customer and one-off tasks are common), if a candidate were to write a python script for the example line of bash I would be strongly biased towards rejecting them.

squeaky-clean 8y ago

That's just one personal anecdote though. I can do the Python example from memory but not the bash example.

Lio 8y ago

One nice thing about using a pipeline is that it will be multi-threaded by default. That is each process will have it's own thread(s) and distribute any work across multiple cores. Also with something like Gnu Parallel you can easily scale across multiple machines. Maybe there are more efficient approaches but a pipeline and gnu parallel is pretty easy to scale up without thinking much about it.