5 ms·

> This belittles their and anyone else's bash scripts for git as 'toys'. That's unfair, and completely unnecessary for your point. I don't think so. It is a ve

by vog 9y ago

> This belittles their and anyone else's bash scripts for git as 'toys'. That's unfair, and completely unnecessary for your point. I don't think so. It is a very common mistake to miss the point where you should switch from a shell script to Python. (... or Ruby, Haskell, OCaml, or any other language with good libraries, sane error handling and sane data structures. And yes, I'm saying this being aware of, and always using, "set -eu -o pipefail".) In other words: A complex collection of shell code is a very good warning indicator that somebody missed that point.

yorwba 9y ago
Using a thin layer of bash to feed the output of a command into a short Python script is an excellent use of shell in my opinion.
- Karunamon 9y ago
  But why would you, when you can just do subprocess.popen on the command and then slice and dice the result in a language that can do regexes, splitting, format strings, and so forth? I mean, if you're going to be using Python anyways, you might as well just use the standard library and skip the extra bash step entirely. The code might be a bit longer, but infinitely safer and more maintainable.
  - yorwba 9y ago
    If all you're doing is rewriting the pipelines you'd have written in Bash using subprocess in Python, you're not making your code safer or more maintainable. Unless I'm missing something, subprocess.Popen requires you to explicitly wire up the inputs and outputs, whereas Bash has clear and readable syntax for the most common idioms. For a program that mostly just calls other programs with the right parameters, Python only obfuscates the important bits with details that a shell abstracts away. For a program that has to do some non-trivial processing, pure Bash+coreutils definitely loses to a general-purpose programming language with good standard library. But that doesn't mean that you have to completely switch over. You can leave the parts in Bash that are just pipelines of other programs (again, Python can't improve anything there) and do your further processing in Python. Whether that's implemented as a Bash script calling Python in a pipeline or Python delegating part of it's processing to a Bash script just depends on the most natural entry point. It might feel "impure" to mix two different languages in a program, but I think using each language in the domain where it's strengths shine is a much better idea than attempting to emulate the core features of one language in the other.
    - mbrock 9y ago
      Often it's a really nice and interesting exercise to approach a problem by thinking along the lines of: "This shell script is a bit complex. Instead of rewriting the shell script in another language, is there some simple and standalone way I could massage the data, in order to make the shell script simple again?" It's quite a lot like thinking like this: "This Python script is getting a bit slow. Instead of rewriting it in a faster language, is there some little part of it that I could isolate and only write that in a faster language?"
falsedan 9y ago
> I don't think so. You think it's necessary to let sysadmins know their place as "not real developers"? That's elitist.
- vog 9y ago
  > That's elitist Wait, what?! I do think that every "sysadmin" who is able to write super-complex shell scripts with large sed/awk scripts is also able to learn Python or Ruby. And I'm convinced that this knowledge will enable them to produce rock-solid tools (which they need!) more easily than with Shell scripts. Splitting the world into admins and programmers - that is elitist. You won't find this distinction in small companies, and you find that in large companies only because that's how they happen to be organized - and are heavily criticized for that, mostly by Agile and DevOps movements. Moreover, removing that distiction is one of the core values of the Free Software movement. The early essays of RMS show this in a very clear way. Many people are doing "real programming" without noticing, and this is why they are using improper tools for that, and should start to learn at least one general-purpose language to make things easier for themselves. It is all about using the right tool for the job, and for complex jobs, a generic programming language with proper libraries is the right tool - far better suited than a shell script gluing together highly specialized tools, which run in separate processes and communicate via ad-hoc text formats rather than proper data structures. For every complex system (OS, games, 3D modelling tools), the emerging power users are also beginning programmers, and the more complex things you are able to build within one DSL or (embeeded) scripting language, the easier it is to jump to a general-purpose programming language. Some systems are so nice that they use a general-purpose programming language for scripting in the first place - such as Lua for many game engines, Python for Blender and Gimp, Scheme (Guile) for Guix package building, and so on.
  - qznc 9y ago
    My sysadmin at the university has explicit orders "not to program". Ironically, he is allowed to write configuration, e.g. Ansible playbooks. So I guess, the trick is to use a turing-complete configuration language which is "totally not programming".
  - 9y ago

by vog 9y ago

yorwba 9y ago

Using a thin layer of bash to feed the output of a command into a short Python script is an excellent use of shell in my opinion.

Karunamon 9y ago
But why would you, when you can just do subprocess.popen on the command and then slice and dice the result in a language that can do regexes, splitting, format strings, and so forth? I mean, if you're going to be using Python anyways, you might as well just use the standard library and skip the extra bash step entirely. The code might be a bit longer, but infinitely safer and more maintainable.
- yorwba 9y ago
  If all you're doing is rewriting the pipelines you'd have written in Bash using subprocess in Python, you're not making your code safer or more maintainable. Unless I'm missing something, subprocess.Popen requires you to explicitly wire up the inputs and outputs, whereas Bash has clear and readable syntax for the most common idioms. For a program that mostly just calls other programs with the right parameters, Python only obfuscates the important bits with details that a shell abstracts away. For a program that has to do some non-trivial processing, pure Bash+coreutils definitely loses to a general-purpose programming language with good standard library. But that doesn't mean that you have to completely switch over. You can leave the parts in Bash that are just pipelines of other programs (again, Python can't improve anything there) and do your further processing in Python. Whether that's implemented as a Bash script calling Python in a pipeline or Python delegating part of it's processing to a Bash script just depends on the most natural entry point. It might feel "impure" to mix two different languages in a program, but I think using each language in the domain where it's strengths shine is a much better idea than attempting to emulate the core features of one language in the other.
  - mbrock 9y ago
    Often it's a really nice and interesting exercise to approach a problem by thinking along the lines of: "This shell script is a bit complex. Instead of rewriting the shell script in another language, is there some simple and standalone way I could massage the data, in order to make the shell script simple again?" It's quite a lot like thinking like this: "This Python script is getting a bit slow. Instead of rewriting it in a faster language, is there some little part of it that I could isolate and only write that in a faster language?"

falsedan 9y ago

> I don't think so. You think it's necessary to let sysadmins know their place as "not real developers"? That's elitist.

vog 9y ago
> That's elitist Wait, what?! I do think that every "sysadmin" who is able to write super-complex shell scripts with large sed/awk scripts is also able to learn Python or Ruby. And I'm convinced that this knowledge will enable them to produce rock-solid tools (which they need!) more easily than with Shell scripts. Splitting the world into admins and programmers - that is elitist. You won't find this distinction in small companies, and you find that in large companies only because that's how they happen to be organized - and are heavily criticized for that, mostly by Agile and DevOps movements. Moreover, removing that distiction is one of the core values of the Free Software movement. The early essays of RMS show this in a very clear way. Many people are doing "real programming" without noticing, and this is why they are using improper tools for that, and should start to learn at least one general-purpose language to make things easier for themselves. It is all about using the right tool for the job, and for complex jobs, a generic programming language with proper libraries is the right tool - far better suited than a shell script gluing together highly specialized tools, which run in separate processes and communicate via ad-hoc text formats rather than proper data structures. For every complex system (OS, games, 3D modelling tools), the emerging power users are also beginning programmers, and the more complex things you are able to build within one DSL or (embeeded) scripting language, the easier it is to jump to a general-purpose programming language. Some systems are so nice that they use a general-purpose programming language for scripting in the first place - such as Lua for many game engines, Python for Blender and Gimp, Scheme (Guile) for Guix package building, and so on.
- qznc 9y ago
  My sysadmin at the university has explicit orders "not to program". Ironically, he is allowed to write configuration, e.g. Ansible playbooks. So I guess, the trick is to use a turing-complete configuration language which is "totally not programming".
- 9y ago