6 ms·

Man, this sounds right, but I dunno ... I feel like even "simple" shell scripts tend toward more inscrutability than all but the most questionable python code I

Man, this sounds right, but I dunno ... I feel like even "simple" shell scripts tend toward more inscrutability than all but the most questionable python code I've ever written. Just comparing my dotfiles - every bit of which I understood when I wrote them - to the gnarliest sections of a big python app I work on ... I just really feel like at least some of this inscrutability issue can truly be laid at the feet of shell / bash as a language.

clnhlzmn 3y ago
This is totally true and the bash apologists are delusional.
- lobocinza 3y ago
  It depends on the use case. Bash code can be elegant, Python code can be ugly. I'm not saying those are the average cases but complex code regardless of the language often is ugly even with effort to make it more readable.
  - clnhlzmn 3y ago
    Oh I get it. Sometimes bash is the right tool for the job. I think that’s just mostly an unfortunate historical artifact though. It’s hard to argue it’s intuitive or “clean” or ${POSITIVE_ADJECTIVE} in the average case.
- TeMPOraL 3y ago
  So are some Python advocates, too. The thing that's worse than a bash script made of Perlish line noise, is a piece of "clean code" dead simple Python that's 80% __language__.boilerplate, 20% logic, smeared over 10x the lines because small functions calling small functions are cool. No one has enough working memory to keep track of what's going on there. Instead, your eyes glaze over it and you convince yourself you understand what's going on. Also, Python build scripts can be living hell too, full of dancing devils that could be introducing backdoors left and right - just look at your average Conan recipe, particularly for larger/more sensitive libraries, like OpenSSL or libcurl.
  - _a_a_a_ 3y ago
    That's a lot of emotive language, can you actually link to some actual examples.
    - TeMPOraL 3y ago
      I mentioned Conan recipes, didn't I? :). Those are my most recent sources of frustration.
      - _a_a_a_ 3y ago
        I've never heard of a conan language, and a couple of URLs to some bad recipes would not go amiss.
        TeMPOraL 3y ago
        Conan is a package manager for C/C++, written in Python. See: https://conan.io/ https://conan.io/. The way it works is that you can provide "recipes", which are Python scripts, that automate the process of collecting source code (usually from a remote Git repository, or a remote source tarball), patching it, making its dependencies and transitive dependencies available, building for specific platform and architecture (via any number of build systems), then packaging up and serving binaries. There's a lot of complexity involved. Here are the two recipes I mentioned: libcurl: https://github.com/conan-io/conan-center-index/blob/master/recipes/libcurl/all/conanfile.py https://github.com/conan-io/conan-center-index/blob/master/r... OpenSSL v3: https://github.com/conan-io/conan-center-index/blob/master/recipes/openssl/3.x.x/conanfile.py https://github.com/conan-io/conan-center-index/blob/master/r... Now, for the sake of this thread I want to highlight three things here: - Conan recipes are usually made by people unaffiliated with the libraries they're packaging; - The recipes are fully Turing-complete, do a lot of work, have their own bugs - therefore they should really be treated as software comonents themselves, for the purpose of OSS clearing/supply chain verification, except as far as I know, nobody does it; - The recipes can, and do, patch source code and build scripts. There's supporting infrastruture for this built into Conan, and of course one can also do it by brute-force search and replace. See e.g. ZLib recipe that does it both at the same time: https://github.com/conan-io/conan-center-index/blob/7b0ac710d99d15b926edd40b3b07bbc83cb5444f/recipes/zlib/all/conanfile.py#L66 https://github.com/conan-io/conan-center-index/blob/7b0ac710... -- `_patch_sources` does both direct search-and-replace in source files, and applies the patches from https://github.com/conan-io/conan-center-index/tree/master/recipes/zlib/all/patches https://github.com/conan-io/conan-center-index/tree/master/r.... Good luck keeping track of what exact code goes into your program, when using Turing-complete "recipe" programs fetched from the Internet, which fetch your libraries from somewhere else on the Internet.
- egorfine 3y ago
  I'm a bash apologist and I totally stand by your words. It's delusional. Bash totally has to go.
dboreham 3y ago
Shell is evil mainly because it's so old. It's largely unchanged since 1978 and had to run on incredibly resource limited machines. What it could achieve on those machines was amazing. Tradeoffs would be different today, e.g. PowerShell.
- SlightlyLeftPad 3y ago
  A big pet peeve of mine is that shell is written off as evil but the only reason I ever hear is basically a variation of “I don’t understand it, therefore it scares me.” The reality is, unlike _really_ old languages like COBOL or RPG, bash is still literally everywhere, it’s installed on practically every linux machine by default which makes deploying a shell script completely trivial. It’s certainly under appreciated and because it’s ubiquitous, widely used in build processes, there’s a responsibility to learn it. It’s not hard, it’s not a wildly complex language. I don’t think these issues would necessarily be solved at all by waving a hand and replacing it with similarly complex build tools. Bazel, for example, can be a daunting tool to fully grasp. Any tool used should be well understood. Easier said than done of course.
  - Espressosaurus 3y ago
    If a language has more footguns than guns-pointed-at-not-my-feet, I consider that a problem with the language. And shell has a lot of footguns.
    - SlightlyLeftPad 3y ago
      I’d argue that every language is loaded full of foot guns. If you’re encountering those foot guns on a regular basis, it’s an issue with the author. That said, what can help drastically here are well-defined best practices and conventions built into the language which, admittedly, bash really doesn’t have.
      - _a_a_a_ 3y ago
        Can you point to 3 python foot guns, for instance?
bayindirh 3y ago
> I feel like even "simple" shell scripts tend toward more inscrutability than all but the most questionable python code I've ever written. When you use long options in bash (scripts), it becomes very readable, but it's not a widespread practice. I always use long options while writing scripts. Consider these two examples, which is very straightforward: - curl -fsSL $URL | bash - curl --fail --silent --show-error --location $URL | bash The second one almost "talks you through".
- sanderjd 3y ago
  IMO, that's not where the inscrutability comes from. Rather, it is things like many useful (and thus used) constructs being implemented with line noise like `"$(foo)"`, `2>&1 &`, `${foo:-bar}`, etc., finicky control flow syntax, surprising defaults that can be changed within unclear scopes, etc.
  - bayindirh 3y ago
    You're right, but you can also expand these. If not syntactically, by adding temporary variables and comments around these complex parts. It's true that Bash and Perl has one of the most contractible syntax around, but it's not impossible to make it more understandable. However, these parts of codebases are considered "supportive" and treated as second class citizens, and never receives the same love core parts of the codebases enjoy. That's a big mistake IMO. When you make something more readable all around, hiding things becomes harder exponentially.
    - im3w1l 3y ago
      I have seen people write bash scripts with a lot of love and attention trying really hard to pay attention to modern best practices and avoiding known pitfalls and using shellcheck. It still looked like shit imo. I challenge you to find a single easily readable >100 line bash script and link it here (I do think small scripts can be fine).
      - bayindirh 3y ago

clnhlzmn 3y ago

This is totally true and the bash apologists are delusional.

lobocinza 3y ago
It depends on the use case. Bash code can be elegant, Python code can be ugly. I'm not saying those are the average cases but complex code regardless of the language often is ugly even with effort to make it more readable.
- clnhlzmn 3y ago
  Oh I get it. Sometimes bash is the right tool for the job. I think that’s just mostly an unfortunate historical artifact though. It’s hard to argue it’s intuitive or “clean” or ${POSITIVE_ADJECTIVE} in the average case.
TeMPOraL 3y ago
So are some Python advocates, too. The thing that's worse than a bash script made of Perlish line noise, is a piece of "clean code" dead simple Python that's 80% __language__.boilerplate, 20% logic, smeared over 10x the lines because small functions calling small functions are cool. No one has enough working memory to keep track of what's going on there. Instead, your eyes glaze over it and you convince yourself you understand what's going on. Also, Python build scripts can be living hell too, full of dancing devils that could be introducing backdoors left and right - just look at your average Conan recipe, particularly for larger/more sensitive libraries, like OpenSSL or libcurl.
- _a_a_a_ 3y ago
  That's a lot of emotive language, can you actually link to some actual examples.
  - TeMPOraL 3y ago
    I mentioned Conan recipes, didn't I? :). Those are my most recent sources of frustration.
    - _a_a_a_ 3y ago
      I've never heard of a conan language, and a couple of URLs to some bad recipes would not go amiss.
      - TeMPOraL 3y ago
        Conan is a package manager for C/C++, written in Python. See: https://conan.io/ https://conan.io/. The way it works is that you can provide "recipes", which are Python scripts, that automate the process of collecting source code (usually from a remote Git repository, or a remote source tarball), patching it, making its dependencies and transitive dependencies available, building for specific platform and architecture (via any number of build systems), then packaging up and serving binaries. There's a lot of complexity involved. Here are the two recipes I mentioned: libcurl: https://github.com/conan-io/conan-center-index/blob/master/recipes/libcurl/all/conanfile.py https://github.com/conan-io/conan-center-index/blob/master/r... OpenSSL v3: https://github.com/conan-io/conan-center-index/blob/master/recipes/openssl/3.x.x/conanfile.py https://github.com/conan-io/conan-center-index/blob/master/r... Now, for the sake of this thread I want to highlight three things here: - Conan recipes are usually made by people unaffiliated with the libraries they're packaging; - The recipes are fully Turing-complete, do a lot of work, have their own bugs - therefore they should really be treated as software comonents themselves, for the purpose of OSS clearing/supply chain verification, except as far as I know, nobody does it; - The recipes can, and do, patch source code and build scripts. There's supporting infrastruture for this built into Conan, and of course one can also do it by brute-force search and replace. See e.g. ZLib recipe that does it both at the same time: https://github.com/conan-io/conan-center-index/blob/7b0ac710d99d15b926edd40b3b07bbc83cb5444f/recipes/zlib/all/conanfile.py#L66 https://github.com/conan-io/conan-center-index/blob/7b0ac710... -- `_patch_sources` does both direct search-and-replace in source files, and applies the patches from https://github.com/conan-io/conan-center-index/tree/master/recipes/zlib/all/patches https://github.com/conan-io/conan-center-index/tree/master/r.... Good luck keeping track of what exact code goes into your program, when using Turing-complete "recipe" programs fetched from the Internet, which fetch your libraries from somewhere else on the Internet.
egorfine 3y ago
I'm a bash apologist and I totally stand by your words. It's delusional. Bash totally has to go.

dboreham 3y ago

Shell is evil mainly because it's so old. It's largely unchanged since 1978 and had to run on incredibly resource limited machines. What it could achieve on those machines was amazing. Tradeoffs would be different today, e.g. PowerShell.

SlightlyLeftPad 3y ago
A big pet peeve of mine is that shell is written off as evil but the only reason I ever hear is basically a variation of “I don’t understand it, therefore it scares me.” The reality is, unlike _really_ old languages like COBOL or RPG, bash is still literally everywhere, it’s installed on practically every linux machine by default which makes deploying a shell script completely trivial. It’s certainly under appreciated and because it’s ubiquitous, widely used in build processes, there’s a responsibility to learn it. It’s not hard, it’s not a wildly complex language. I don’t think these issues would necessarily be solved at all by waving a hand and replacing it with similarly complex build tools. Bazel, for example, can be a daunting tool to fully grasp. Any tool used should be well understood. Easier said than done of course.
- Espressosaurus 3y ago
  If a language has more footguns than guns-pointed-at-not-my-feet, I consider that a problem with the language. And shell has a lot of footguns.
  - SlightlyLeftPad 3y ago
    I’d argue that every language is loaded full of foot guns. If you’re encountering those foot guns on a regular basis, it’s an issue with the author. That said, what can help drastically here are well-defined best practices and conventions built into the language which, admittedly, bash really doesn’t have.
    - _a_a_a_ 3y ago
      Can you point to 3 python foot guns, for instance?

bayindirh 3y ago

> I feel like even "simple" shell scripts tend toward more inscrutability than all but the most questionable python code I've ever written. When you use long options in bash (scripts), it becomes very readable, but it's not a widespread practice. I always use long options while writing scripts. Consider these two examples, which is very straightforward: - curl -fsSL $URL | bash - curl --fail --silent --show-error --location $URL | bash The second one almost "talks you through".

sanderjd 3y ago
IMO, that's not where the inscrutability comes from. Rather, it is things like many useful (and thus used) constructs being implemented with line noise like `"$(foo)"`, `2>&1 &`, `${foo:-bar}`, etc., finicky control flow syntax, surprising defaults that can be changed within unclear scopes, etc.
- bayindirh 3y ago
  You're right, but you can also expand these. If not syntactically, by adding temporary variables and comments around these complex parts. It's true that Bash and Perl has one of the most contractible syntax around, but it's not impossible to make it more understandable. However, these parts of codebases are considered "supportive" and treated as second class citizens, and never receives the same love core parts of the codebases enjoy. That's a big mistake IMO. When you make something more readable all around, hiding things becomes harder exponentially.
  - im3w1l 3y ago
    I have seen people write bash scripts with a lot of love and attention trying really hard to pay attention to modern best practices and avoiding known pitfalls and using shellcheck. It still looked like shit imo. I challenge you to find a single easily readable >100 line bash script and link it here (I do think small scripts can be fine).
    - bayindirh 3y ago