14 ms·

Getting anything out of a subshell that isn't from STDOUT is impossible. So you can't define an array in a subshell and then use it outside the subshell, and yo

Getting anything out of a subshell that isn't from STDOUT is impossible. So you can't define an array in a subshell and then use it outside the subshell, and you can't return an array (or anything that isn't a string) from a subshell. If you only use subshells and want to use any kind of data structure that isn't a string passed from STDOUT, you have to do it globally. And subshells are slow. So nobody uses subshells. If you use Bash for programming, you have to stop thinking in terms of the holier-than-thou software engineer, whose ego believes that a superior, "clean" design makes a superior program. You should embrace globals. You should switch between using or not using the enforcement of set variables or program exit status. You should stop using Bashisms and subtle, obscure language features unless you absolutely have to. Bash is not a "real" programming language, so do not treat it as one. Do not look for hidden features, or try to do things in cute ways that nobody else uses. There is no superior method or hidden knowledge. Just write extremely simple code and understand the quirks of the shell.

1vuio0pswjnm7 5y ago
"So nobody uses subshells." djb has always used them, even in the shortest of scripts. Otherwise I agree with everything in this comment. There is a reasonable argument to avoid using bash for non-interactive scripts. The benefits of any additional bash features, so-called "bashisms", arguably do not outweigh the costs of making these scripts non-portable. One of the many advantages of shell scripts is that they are portable and have tremendous longevity; shell scripts can last a long, long time. There are no version changes and aggressive feature creep to worry about as is routinely the case with programming languages. The scripts just keep working, every day, and we can forget we are even using them, e.g., they are being used in the various ways by UNIX-like OS distributions. One of the today's programmer memes was "Get Stuff Done". Maybe the shell was not meant for today's programmers. But for "sysadmins", or "DevOps", or whatever term anyone comes up with in the future, people who can "administer/operate" computers running a UNIX-like OS for themselves or for someone else like a client or an employer, the Bourne shell works for its intended purpose, better than anything anyone has come up with in the last 50+ years. There's a lot of stuff that "gets done" with the shell, whether it is on someone's own computer, their client's or their employer's. Attempts at "shell replacements", cf. alternative shells, usually look like interpreters for programming languages, not shells. Perhaps this is not a coincidence. Bourne shell is relatively small and fast. Computers with small form factors often use UNIX-like OS and when they do they usually include shells. Many programmers seem to dislike the notion that such a layer exists. They often try to blame the shell instead of their own lack of interest in learning to use it. The shell is boring, and "boring" is sometimes the wisest choice. Most software has expanded to consume available resources (often the developer's choice not the user's). This makes computer performance gains difficult for the end user to discern, e.g., decade after decade, routine tasks seem to take the same amount of time. However the shell and many "standard" UNIX utilities have not changed as much in the same period. Subshells may have been "slow" many years ago. IMHO, they do not feel slow today. Routine tasks performed with the shell seem to run faster, as one would expect after hardware upgrade. I posit: Life is too short to learn every programming language du jour but it is long enough to learn the Bourne shell, reasonably well.
- oweiler 5y ago
  Bash scripts and shell scripts are not portable. They have to shell out to other commands in order to do something meaningful. Depending on your OS, these commands may not exist, or have a different set of commandline flags.
- zsz 5y ago
  One of the principal things that "got done" using Bash (at least until 2017 -- after that, I couldn't say) off the top of my head was the GFS at NCEP. Yes, it was predominantly Fortran for number crunching; but what tied all the individual programs together was a super massive, maintained, and constantly modified and improved Bash script. And yes, there were plans in the works to switch out for something else (which is why I cannot say, four years later, what the status quo is). I should also mention, it was this Bash script that ran parallelized on the supercomputer -- and it was ultimately this script which, every three hours, produced the next set of forecasts, 24/7.
- 1vuio0pswjnm7 5y ago
  "Shell" as used here means the Bourne POSIX-like shell, such as NetBSD's Almquist shell, not userland utilities that the shell may or may not call in a script. Scripts can of course test for differences in utilities where there is uncertainty. If scripts written for a POSIX-like "lowest common denominator" shell were not portable, methinks projects written for such a shell, such as autoconf, would not work on so many diffferent systems. The amount of free software found in open source OS repositories that is built using "configure" shell scripts is not small. Debian and other Linux distributions use a shell derived from NetBSD's Almquist shell. People who design and maintain these operating systems suggest that, for non-interactive use, this Bourne shell is faster than Bash.
kristianpaul 5y ago
No need for array when i can create files and combine their content with commands like tac and other coreutils text utilities.
fiddlerwoaroof 5y ago
If you take the time and effort to understand how bash wants you to think, you can learn how to right elegant scripts that are as maintainable as anything else.
- hutrdvnj 5y ago
  But in reality you never see such scripts.
- jefftk 5y ago
  I've written a lot in bash over the years, and I feel like I understand it pretty well. But I would never say that even elegant bash scripts are as maintainable as "anything else". It is a clunky programming environment, born of compromises, with many traps that are easy to miss in code review.
  - tempodox 5y ago
    There's `shellcheck`, as part of Syntastic, for vim. I find it quite useful. You could call it a Bash linter.
    - pletnes 5y ago
      Shellcheck is great and is a standalone cli program, as well as a website if you don’t want to install it locally. https://www.shellcheck.net/ https://www.shellcheck.net/
      - lazyweb 5y ago
        I'd rather always check my code locally if the alternative is pasting it into a random web from. On the other hand, I wonder if the web page keeps track and could offer "best of bash mistakes".
        naniwaduni 5y ago
        A random web form can exfiltrate the data you paste into it (and whatever your browser lets it gather). A local program can exfiltrate ... approximately everything of value on the machine?
ReactiveJelly 5y ago
Just don't use Bash.
- gavinray 5y ago
  A lot of the time you don't even have the luxury of assuming "bash" is present, you've got only "sh". Since all valid "sh" syntax is valid "bash", you've got to restrict yourself to only sh-valid code in the event the host doesn't have bash. I have gotten used to doing this -- these scripts usually wind up being run in Docker containers that don't have "bash" installed for the sake of minimalism. IE, you can't use [[ $predicate ]], but instead [ $predicate ], etc. Lot of subtle differences.
  - Spivak 5y ago
    Not all sh syntax is valid bash — in bash [[ is reserved but not in sh. And if you want syntax that is valid but doesn’t do the same thing in sh vs bash there’s plenty of examples.
  - kevin_thibedeau 5y ago
    You can't ever assume that /bin/sh is a POSIX shell so then you get limited to just an early Bourne subset.
- mistrial9 5y ago
  I would agree with this and use python instead, but for uuhh "python problems with strings". IF you are 'blue sky' open, making new things, yes, sure. no BASH. But for things I built 8 years ago that are non-trivial, and I don't prioritize an entire rewrite, BASH works very well. And guess what, every two years that 8-years-ago just comes along with it. BASH is here to stay.
- chasil 5y ago
  The POSIX shell can sometimes achieve with a few lines what would take dissertations to do in C. The supersets of POSIX are often even more effective. Ugly as the tool may be, in the box it stays.
laumars 5y ago
> Bash is not a "real" programming language, so do not treat it as one. Bash is definitely a different paradigm to your average imperative or functional language and thus requires a different approach but I wouldn’t go so far as to say it’s not “real”. It certainly fits the criteria of a programming language even if it does have more warts than a pantomime witch.
- ratww 5y ago
  Yes. Compared to other languages, Bash can be incredibly worse for some things, but much better for others. People who don't realise that won't be able to get the best out of their toolset.
- craftinator 5y ago
  It is Turing Complete...
  - laumars 5y ago
    I don’t think it makes much sense to judge software by whether it is Turing complete because otherwise that would make Minecraft a programming language
    - craftinator 5y ago
      So are you deprecating the term "Turing Complete"? Mighty bold move there. And on that same note, have you seen the stuff being done in Minecraft? It is quite up for debate whether it is a programming language.
      - laumars 5y ago
        > So are you deprecating the term "Turing Complete"? No. It’s still an interesting yard stick. It just doesn’t describe programming languages in full. For example, and someone counterintuitively, it is actually possible to design a programming language that isn’t Turing complete. Some esoteric functional languages do actually fall into that category. But generally programming languages would be a subset of Turing complete software. > And on that same note, have you seen the stuff being done in Minecraft? I have, hence why I cited it as an example. > It is quite up for debate whether it is a programming language. One could debate anything but it doesn’t mean the argument is made in good faith. Minecraft is a game. I’m happy to even extend the definition and say it’s supports some visual programming mechanics. But that doesn’t mean it is a programming language. Just because something has 4 legs and a tail, it doesn’t automatically mean it is a dog.
worrycue 5y ago
> Bash is not a "real" programming language, so do not treat it as one. Is there a reason we aren’t using a shell with a proper programming language for scripting? I can’t say I’m a fan of bashscript. I wonder why we are using this weird language.
- jrochkind1 5y ago
  Because cross-OS (not just cross-linux-distro but even that) standards-making doesn't exist anymore in a real non-broken way, and we're stuck with whatever standards of 20+ years ago. Whether official or de facto (technically bash is just de facto although `bash --posix` or `sh` is an official real standard). There basically isn't any "innovation" happening in this space in a way that could really result in anything as pervasive as bash. Maybe also cause it's just not "interesting" anymore, the 'sexy' things are many levels of abstraction higher than they were when bash came to dominance. It feels like now we're just stuck with it forever, indeed.
  - m3at 5y ago
    I agree with your broader point that we seem stuck with old shells and no contenders seems in a position to replace them. However some are certainly trying, like osh [1] that takes backwards compatibility with bash seriously, which makes adoption easier. Maybe it'll be like the C++ vs Rust situation? https://www.oilshell.org/ https://www.oilshell.org/
  - quotemstr 5y ago
    Maybe it's time for some other body --- say, freedesktop.org --- to start standardizing things that have traditionally been in the domain of POSIX. After all, HTML got much better when WHATWG took W3C's toys away and actually started innovating HTML again. Maybe it's time to do something similar with Unix: I'm sick and tired of people telling everyone that we have to stay in some 1995 time capsule because we need to follow The Standard and if The Standard hasn't changed, too bad.
  - majormajor 5y ago
    You say "don't exist anymore" but when was it better? Pre-Mac-OS-X you didn't even have an easy to access shell there on Macs, and Windows wasn't bash either. So if there was a golden era for that, it never included the two predominant desktop OSes. De-facto standardization has at least brought us free officially-supported options for running Bash or other *nix shells on both of those platforms(if terribly outdated on Mac).
michaelcampbell 5y ago
> So nobody uses subshells. I'll count myself among nobodies, I guess. I use them a fair bit when part of said subshell cd's somewhere, and I don't feel like using pushd/popd, either of which might fail to operate based on my mistakes, but subshells seem to never fail to exit, eventually.
- dotancohen 5y ago
  Depends what you're doing. If you need the exit status of something _in_ that subshell, or even just its STDOUT, then you're stuck.
gpderetta 5y ago
Well you can use arbitrary file descriptors to get things out of a subshell, not just stdout. I had to do it when I had to get something out of band, while redirecting stdout through a pipeline. Mind, if possible, it is even uglier than using stdout.
- cryptonector 5y ago
  Few people know about `{varname}` I/O redirection...
  - gpderetta 5y ago
    Yes. The fact that shell scripting is both so powerful and yet such an ugly language is painful.
- Groxx 5y ago
  yeah, I was wondering if there was some reason that something like this wouldn't work: err_f="$(mktemp)" third_f="$(mktemp)" out="$(cmd 2>"$err_f" 3>"$third_f")" err="$(cat "$err_f")" third="$(cat "$third_f")" Seems like that should work fine, and it's not even that odd looking or hard to understand. But I haven't tried, not sure if there's some surprise lurking in the depths of tempfiles or something.
  - gpderetta 5y ago
    Normally I use mkfifo then immediately rm it after opening the file with exec. The fact that there isn't a good way to open an anonymous pipe and get its two file descriptors in bash is very annoying.
arendtio 5y ago
throwaway984393 <-- just a troll... Edit: It seems I should elaborate. > Getting anything out of a subshell that isn't from STDOUT is impossible. So you can't define an array in a subshell and then use it outside the subshell, and you can't return an array (or anything that isn't a string) from a subshell. If you only use subshells and want to use any kind of data structure that isn't a string passed from STDOUT, you have to do it globally. Yes, but that is no problem at all, because that is the way shell scripts work and if you do it in a functional way, where the function has no state, it is great (we a few exceptions). > And subshells are slow. So nobody uses subshells. Shell scripts are slow in general, subshells don't make an exception but also don't have a large impact. And people do use subshells. > If you use Bash for programming, you have to stop thinking in terms of the holier-than-thou software engineer, whose ego believes that a superior, "clean" design makes a superior program. You should embrace globals. You should switch between using or not using the enforcement of set variables or program exit status. You should stop using Bashisms and subtle, obscure language features unless you absolutely have to. If you use Shell scripts, you should understand that this language has been designed decades ago and that professionals advise to use it just in short scripts to connect binaries. > Bash is not a "real" programming language, so do not treat it as one. Do not look for hidden features, or try to do things in cute ways that nobody else uses. There is no superior method or hidden knowledge. Just write extremely simple code and understand the quirks of the shell. That part is mostly okay, but "real" doesn't get the point, as it is a real language, just one with many problems. However, given its superior strength we haven't overcome it yet...
- fstrthnscnd 5y ago
  > However, given its superior strength we haven't overcome it yet... "Superior strength" is probably just a matter taste, however I would really like to read your explanation on why you think so.
  - arendtio 5y ago
    Well, if it would be just bad, why do we still use it? Why don't we use e.g. Python (which is a much better designed programming language)? Some reasons why POSIX shells are great: - they are part of POSIX (a Standard which is implemented by multiple OS) and therefore often available - you can use them to compose programs written in different languages - you can easily connect the output of one program with the input of another (pipes)
gfodor 5y ago
This reminds me of the arguments made against writing “real” code in JavaScript in the early days of the web, until Crockford came along and wrote “The Good Parts.” There is no reason to think that a few idioms and curating features could go a long way to leading to a much better, less hacky paradigm for bash shell scripting.
- hsn915 5y ago
  Bash is much older than JavaScript. If it was going to turn into a real programming language, it would have by now. It hasn't. Also, JavaScript really is not a very good programming language. We are just stuck with it because it's the only language the browser understands. (Well, until recently: things are going to change with the introduction of wasm). But for the shell, we're not stuck with any one language. Whatever you want to do can be programmed in your favorite language. You can easily write a python script instead of a bash function.
  - raziel2p 5y ago
    > things are going to change with the introduction of wasm people have been saying this for years already and nothing seems to have come of it :(
    - tambourine_man 5y ago
      Yeah. Turns out, having an interpreted runtime in the browser is quite useful and sending bytecode across the wire not as practical. We've been forgetting and relearning that for the past two decades.
      - hsn915 5y ago
        "turns out"? How did it turn out so? Actually it turns out that having an interpreted runtime causes a stream of endless problem. We keep try to patch them over but new problems keep coming up. It turns out that a standarized low level byte code is the right way to ship cross platform applications. We already knew that. It just took a long time to go through the standarization process and be implemented by browsers. If it seems like it's not in use today, it's largely because JavaScript has momentum and the majority of web programmers don't yet know how to take advantage of wasm (or maybe don't think they need it).
necheffa 5y ago
LOLWUT? At least _try_ to understand what you are talking about before you criticize. Your rant demonstrates your complete lack of understanding on the topic. The, apparently, "hidden" knowledge is that Unix shells are designed to process text. And so you serialize complex data structures over FIFOs from child to parent. That can be as simple as comma/colon/whatever delimited bytestreams chopped up with awk or as complex as JSON and parsed with jq.
- quotemstr 5y ago
  Sometimes we want to write functions that have side effects. Subshell functions don't have side effects. Sometimes you don't want side effects, but sometimes you do, and it's silly to completely foreclose on even the possibility of using side effects.
  - chaps 5y ago
    Isn't grep your friend in this situation? I'm really not sure why you can't have side effects in this scenario.
    - quotemstr 5y ago
      What on earth does grep have to do with side effects?
      - chaps 5y ago
        And what makes you think side effects aren't possible in the model described? Think of grep as a way to control flow. If one side effect happens, send to stdout one thing. Another side effect, send something else. Then iterate over those outputs.
    - rablackburn 5y ago
      A “side-effect” in this case is the ability for a function you call to do something “outside” of itself. For example, to change the value of a global variable. It’s generally considered best practice (in the functional programming community at least) to write “pure” functions (that is, functions without side effects) because it’s much easier to reason about what they are doing. So good news, subshells can _only_ be pure (the only way to get anything back from them is if they write some string to stdout), but sometimes you do actually want to have some side-effects (imagine a function that reads a config file and then wants to set some of the global variables to the values found there). Well, bad luck. If you use the subshell syntax you literally can’t do that.
zsz 5y ago
Sir! Yes, sir!
quotemstr 5y ago
You're right about the disadvantages of subshells, but wrong about this: > avoid bashisms What's holier-than-thou is insisting that people stick to a dialect of shell scripting that hasn't changed in decades because some people think that conforming to POSIX is its own reward even if it makes life harder and programs worse. No, thanks. I'll stick to bashisms. Things like coprocesses and filemap make certain classes of program much easier to write. And yes, shell scripts are programs, and the quickest way to turn these programs into unmaintainable balls of mud is to follow your advice to avoid normal programming best practices simply because the program you're writing happens to be a shell script.
presto8 5y ago
Here is a script that shows exporting results from a subprocess to a parent shell using a temporary file: #!/usr/bin/env bash process_line() { if [[ $* == *foo* ]]; then echo "found_foo=1" >>$IPC fi } IPC=$(mktemp) find . -maxdepth 1 -type f | while read line; do # note: this is run a subprocess process_line "$line" done source "$IPC" echo "found_foo=$found_foo"
rhizome 5y ago
If you want to program, use a programming language.
tomjakubowski 5y ago
> So you can't define an array in a subshell and then use it outside the subshell, and you can't return an array (or anything that isn't a string) from a subshell Huh. How does one return a value from a bash function? I've always only ever seen the "echo calling convention" used in scripts, or use of global variables.
- kps 5y ago
  Bash does have `typeset -n` (from ksh) which allows shell variables to be passed to a function by reference.
  - cryptonector 5y ago
    Yes, and which gets you Tcl-ish pass by name semantics.
  - tomjakubowski 5y ago
    Cool, thanks!
cryptonector 5y ago
> Getting anything out of a subshell that isn't from STDOUT is impossible. You can use I/O redirection with arbitrary file descriptor numbers, and you can let the shell pick them. So you can `mknod` a pipe, `exec {pipefd_rw}<>"$pipe_name" {pipefd_ro}<"$pipe_name" {pipefd_rw}>"$pipe_name";` and now you have three open file descriptors that you can let the sub-shell use to communicate with the parent. And so you're not limited to stdin/stdout. I do wish there was a built-in for making an anonymous pipe instead of having to make one on filesystem. (You have to open the named pipe for read-write first because otherwise you'll block, but if you don't want a read-write FD for it you can close it after opening the read and write FDs.)
foxes 5y ago
Yep bash is not a real language. If anything you are saying it should be deprecated in place for a real language. If people want to do more advanced things why does the basic language of their shell not support it? Why should you not care about the correctness of your shell scripts which can run important things? Seems a perfect place for a functional language. It’s high level, you can just declare what you want. I imagine your interpreter can add in all sorts of checks to help make your scripting more correct.
- addicted 5y ago
  Powershell attempts to use a more scripting friendly language in the shell. And it’s a horrible shell experience. It’s great for scripting (but not as good as other scripting languages) but it’s a terrible experience on a day to day basis, with the only salvation being the existence of 2-3 letter aliases which make things slightly more manageable. Bash is a great shell language that whose scripts are useful because it allows you to trivially convert your manual shell actions into an executable script. If you need to do anything complex (or rather, something you probably wouldn’t type into the shell manually if you had to do the thing on a one off basis), then you’re probably better off using a different scripting language. But changing BASH with scripting as a priority would probably make it a worse shell language (which is not to say there are no improvements to be made…there are).
dotancohen 5y ago
> Getting anything out of a subshell that isn't from STDOUT is impossible. Yes that is a known limitation that has horrible, horrible workarounds. Don't use them. That said, for scripts whose main input _is_ from a file or STDIN, and also whose main output _is_ to a file or STDOUT, then bash is far more often the right tool for the job than it is given credit for. Of course, I'm talking about bash and a host of other utilities that are often packaged with it such as awk, grep, sed, cut, etc. For processing text I find myself often choosing between bash and Python, and very often if I choose bash I'll feel that I've made the right choice.

1vuio0pswjnm7 5y ago

"So nobody uses subshells." djb has always used them, even in the shortest of scripts. Otherwise I agree with everything in this comment. There is a reasonable argument to avoid using bash for non-interactive scripts. The benefits of any additional bash features, so-called "bashisms", arguably do not outweigh the costs of making these scripts non-portable. One of the many advantages of shell scripts is that they are portable and have tremendous longevity; shell scripts can last a long, long time. There are no version changes and aggressive feature creep to worry about as is routinely the case with programming languages. The scripts just keep working, every day, and we can forget we are even using them, e.g., they are being used in the various ways by UNIX-like OS distributions. One of the today's programmer memes was "Get Stuff Done". Maybe the shell was not meant for today's programmers. But for "sysadmins", or "DevOps", or whatever term anyone comes up with in the future, people who can "administer/operate" computers running a UNIX-like OS for themselves or for someone else like a client or an employer, the Bourne shell works for its intended purpose, better than anything anyone has come up with in the last 50+ years. There's a lot of stuff that "gets done" with the shell, whether it is on someone's own computer, their client's or their employer's. Attempts at "shell replacements", cf. alternative shells, usually look like interpreters for programming languages, not shells. Perhaps this is not a coincidence. Bourne shell is relatively small and fast. Computers with small form factors often use UNIX-like OS and when they do they usually include shells. Many programmers seem to dislike the notion that such a layer exists. They often try to blame the shell instead of their own lack of interest in learning to use it. The shell is boring, and "boring" is sometimes the wisest choice. Most software has expanded to consume available resources (often the developer's choice not the user's). This makes computer performance gains difficult for the end user to discern, e.g., decade after decade, routine tasks seem to take the same amount of time. However the shell and many "standard" UNIX utilities have not changed as much in the same period. Subshells may have been "slow" many years ago. IMHO, they do not feel slow today. Routine tasks performed with the shell seem to run faster, as one would expect after hardware upgrade. I posit: Life is too short to learn every programming language du jour but it is long enough to learn the Bourne shell, reasonably well.

oweiler 5y ago
Bash scripts and shell scripts are not portable. They have to shell out to other commands in order to do something meaningful. Depending on your OS, these commands may not exist, or have a different set of commandline flags.
zsz 5y ago
One of the principal things that "got done" using Bash (at least until 2017 -- after that, I couldn't say) off the top of my head was the GFS at NCEP. Yes, it was predominantly Fortran for number crunching; but what tied all the individual programs together was a super massive, maintained, and constantly modified and improved Bash script. And yes, there were plans in the works to switch out for something else (which is why I cannot say, four years later, what the status quo is). I should also mention, it was this Bash script that ran parallelized on the supercomputer -- and it was ultimately this script which, every three hours, produced the next set of forecasts, 24/7.
1vuio0pswjnm7 5y ago
"Shell" as used here means the Bourne POSIX-like shell, such as NetBSD's Almquist shell, not userland utilities that the shell may or may not call in a script. Scripts can of course test for differences in utilities where there is uncertainty. If scripts written for a POSIX-like "lowest common denominator" shell were not portable, methinks projects written for such a shell, such as autoconf, would not work on so many diffferent systems. The amount of free software found in open source OS repositories that is built using "configure" shell scripts is not small. Debian and other Linux distributions use a shell derived from NetBSD's Almquist shell. People who design and maintain these operating systems suggest that, for non-interactive use, this Bourne shell is faster than Bash.

kristianpaul 5y ago

No need for array when i can create files and combine their content with commands like tac and other coreutils text utilities.

fiddlerwoaroof 5y ago

If you take the time and effort to understand how bash wants you to think, you can learn how to right elegant scripts that are as maintainable as anything else.

hutrdvnj 5y ago
But in reality you never see such scripts.
jefftk 5y ago
I've written a lot in bash over the years, and I feel like I understand it pretty well. But I would never say that even elegant bash scripts are as maintainable as "anything else". It is a clunky programming environment, born of compromises, with many traps that are easy to miss in code review.
- tempodox 5y ago
  There's `shellcheck`, as part of Syntastic, for vim. I find it quite useful. You could call it a Bash linter.
  - pletnes 5y ago
    Shellcheck is great and is a standalone cli program, as well as a website if you don’t want to install it locally. https://www.shellcheck.net/ https://www.shellcheck.net/
    - lazyweb 5y ago
      I'd rather always check my code locally if the alternative is pasting it into a random web from. On the other hand, I wonder if the web page keeps track and could offer "best of bash mistakes".
      - naniwaduni 5y ago
        A random web form can exfiltrate the data you paste into it (and whatever your browser lets it gather). A local program can exfiltrate ... approximately everything of value on the machine?

ReactiveJelly 5y ago

Just don't use Bash.

gavinray 5y ago
A lot of the time you don't even have the luxury of assuming "bash" is present, you've got only "sh". Since all valid "sh" syntax is valid "bash", you've got to restrict yourself to only sh-valid code in the event the host doesn't have bash. I have gotten used to doing this -- these scripts usually wind up being run in Docker containers that don't have "bash" installed for the sake of minimalism. IE, you can't use [[ $predicate ]], but instead [ $predicate ], etc. Lot of subtle differences.
- Spivak 5y ago
  Not all sh syntax is valid bash — in bash [[ is reserved but not in sh. And if you want syntax that is valid but doesn’t do the same thing in sh vs bash there’s plenty of examples.
- kevin_thibedeau 5y ago
  You can't ever assume that /bin/sh is a POSIX shell so then you get limited to just an early Bourne subset.
mistrial9 5y ago
I would agree with this and use python instead, but for uuhh "python problems with strings". IF you are 'blue sky' open, making new things, yes, sure. no BASH. But for things I built 8 years ago that are non-trivial, and I don't prioritize an entire rewrite, BASH works very well. And guess what, every two years that 8-years-ago just comes along with it. BASH is here to stay.
chasil 5y ago
The POSIX shell can sometimes achieve with a few lines what would take dissertations to do in C. The supersets of POSIX are often even more effective. Ugly as the tool may be, in the box it stays.

laumars 5y ago

> Bash is not a "real" programming language, so do not treat it as one. Bash is definitely a different paradigm to your average imperative or functional language and thus requires a different approach but I wouldn’t go so far as to say it’s not “real”. It certainly fits the criteria of a programming language even if it does have more warts than a pantomime witch.

ratww 5y ago
Yes. Compared to other languages, Bash can be incredibly worse for some things, but much better for others. People who don't realise that won't be able to get the best out of their toolset.
craftinator 5y ago
It is Turing Complete...
- laumars 5y ago
  I don’t think it makes much sense to judge software by whether it is Turing complete because otherwise that would make Minecraft a programming language
  - craftinator 5y ago
    So are you deprecating the term "Turing Complete"? Mighty bold move there. And on that same note, have you seen the stuff being done in Minecraft? It is quite up for debate whether it is a programming language.
    - laumars 5y ago
      > So are you deprecating the term "Turing Complete"? No. It’s still an interesting yard stick. It just doesn’t describe programming languages in full. For example, and someone counterintuitively, it is actually possible to design a programming language that isn’t Turing complete. Some esoteric functional languages do actually fall into that category. But generally programming languages would be a subset of Turing complete software. > And on that same note, have you seen the stuff being done in Minecraft? I have, hence why I cited it as an example. > It is quite up for debate whether it is a programming language. One could debate anything but it doesn’t mean the argument is made in good faith. Minecraft is a game. I’m happy to even extend the definition and say it’s supports some visual programming mechanics. But that doesn’t mean it is a programming language. Just because something has 4 legs and a tail, it doesn’t automatically mean it is a dog.

worrycue 5y ago

> Bash is not a "real" programming language, so do not treat it as one. Is there a reason we aren’t using a shell with a proper programming language for scripting? I can’t say I’m a fan of bashscript. I wonder why we are using this weird language.

jrochkind1 5y ago
Because cross-OS (not just cross-linux-distro but even that) standards-making doesn't exist anymore in a real non-broken way, and we're stuck with whatever standards of 20+ years ago. Whether official or de facto (technically bash is just de facto although `bash --posix` or `sh` is an official real standard). There basically isn't any "innovation" happening in this space in a way that could really result in anything as pervasive as bash. Maybe also cause it's just not "interesting" anymore, the 'sexy' things are many levels of abstraction higher than they were when bash came to dominance. It feels like now we're just stuck with it forever, indeed.
- m3at 5y ago
  I agree with your broader point that we seem stuck with old shells and no contenders seems in a position to replace them. However some are certainly trying, like osh [1] that takes backwards compatibility with bash seriously, which makes adoption easier. Maybe it'll be like the C++ vs Rust situation? https://www.oilshell.org/ https://www.oilshell.org/
- quotemstr 5y ago
  Maybe it's time for some other body --- say, freedesktop.org --- to start standardizing things that have traditionally been in the domain of POSIX. After all, HTML got much better when WHATWG took W3C's toys away and actually started innovating HTML again. Maybe it's time to do something similar with Unix: I'm sick and tired of people telling everyone that we have to stay in some 1995 time capsule because we need to follow The Standard and if The Standard hasn't changed, too bad.
- majormajor 5y ago
  You say "don't exist anymore" but when was it better? Pre-Mac-OS-X you didn't even have an easy to access shell there on Macs, and Windows wasn't bash either. So if there was a golden era for that, it never included the two predominant desktop OSes. De-facto standardization has at least brought us free officially-supported options for running Bash or other *nix shells on both of those platforms(if terribly outdated on Mac).

michaelcampbell 5y ago

> So nobody uses subshells. I'll count myself among nobodies, I guess. I use them a fair bit when part of said subshell cd's somewhere, and I don't feel like using pushd/popd, either of which might fail to operate based on my mistakes, but subshells seem to never fail to exit, eventually.

dotancohen 5y ago
Depends what you're doing. If you need the exit status of something _in_ that subshell, or even just its STDOUT, then you're stuck.

gpderetta 5y ago

Well you can use arbitrary file descriptors to get things out of a subshell, not just stdout. I had to do it when I had to get something out of band, while redirecting stdout through a pipeline. Mind, if possible, it is even uglier than using stdout.

cryptonector 5y ago
Few people know about `{varname}` I/O redirection...
- gpderetta 5y ago
  Yes. The fact that shell scripting is both so powerful and yet such an ugly language is painful.
Groxx 5y ago
yeah, I was wondering if there was some reason that something like this wouldn't work: err_f="$(mktemp)" third_f="$(mktemp)" out="$(cmd 2>"$err_f" 3>"$third_f")" err="$(cat "$err_f")" third="$(cat "$third_f")" Seems like that should work fine, and it's not even that odd looking or hard to understand. But I haven't tried, not sure if there's some surprise lurking in the depths of tempfiles or something.
- gpderetta 5y ago
  Normally I use mkfifo then immediately rm it after opening the file with exec. The fact that there isn't a good way to open an anonymous pipe and get its two file descriptors in bash is very annoying.

arendtio 5y ago

throwaway984393 <-- just a troll... Edit: It seems I should elaborate. > Getting anything out of a subshell that isn't from STDOUT is impossible. So you can't define an array in a subshell and then use it outside the subshell, and you can't return an array (or anything that isn't a string) from a subshell. If you only use subshells and want to use any kind of data structure that isn't a string passed from STDOUT, you have to do it globally. Yes, but that is no problem at all, because that is the way shell scripts work and if you do it in a functional way, where the function has no state, it is great (we a few exceptions). > And subshells are slow. So nobody uses subshells. Shell scripts are slow in general, subshells don't make an exception but also don't have a large impact. And people do use subshells. > If you use Bash for programming, you have to stop thinking in terms of the holier-than-thou software engineer, whose ego believes that a superior, "clean" design makes a superior program. You should embrace globals. You should switch between using or not using the enforcement of set variables or program exit status. You should stop using Bashisms and subtle, obscure language features unless you absolutely have to. If you use Shell scripts, you should understand that this language has been designed decades ago and that professionals advise to use it just in short scripts to connect binaries. > Bash is not a "real" programming language, so do not treat it as one. Do not look for hidden features, or try to do things in cute ways that nobody else uses. There is no superior method or hidden knowledge. Just write extremely simple code and understand the quirks of the shell. That part is mostly okay, but "real" doesn't get the point, as it is a real language, just one with many problems. However, given its superior strength we haven't overcome it yet...

fstrthnscnd 5y ago
> However, given its superior strength we haven't overcome it yet... "Superior strength" is probably just a matter taste, however I would really like to read your explanation on why you think so.
- arendtio 5y ago
  Well, if it would be just bad, why do we still use it? Why don't we use e.g. Python (which is a much better designed programming language)? Some reasons why POSIX shells are great: - they are part of POSIX (a Standard which is implemented by multiple OS) and therefore often available - you can use them to compose programs written in different languages - you can easily connect the output of one program with the input of another (pipes)

gfodor 5y ago

This reminds me of the arguments made against writing “real” code in JavaScript in the early days of the web, until Crockford came along and wrote “The Good Parts.” There is no reason to think that a few idioms and curating features could go a long way to leading to a much better, less hacky paradigm for bash shell scripting.

hsn915 5y ago
Bash is much older than JavaScript. If it was going to turn into a real programming language, it would have by now. It hasn't. Also, JavaScript really is not a very good programming language. We are just stuck with it because it's the only language the browser understands. (Well, until recently: things are going to change with the introduction of wasm). But for the shell, we're not stuck with any one language. Whatever you want to do can be programmed in your favorite language. You can easily write a python script instead of a bash function.
- raziel2p 5y ago
  > things are going to change with the introduction of wasm people have been saying this for years already and nothing seems to have come of it :(
  - tambourine_man 5y ago
    Yeah. Turns out, having an interpreted runtime in the browser is quite useful and sending bytecode across the wire not as practical. We've been forgetting and relearning that for the past two decades.
    - hsn915 5y ago
      "turns out"? How did it turn out so? Actually it turns out that having an interpreted runtime causes a stream of endless problem. We keep try to patch them over but new problems keep coming up. It turns out that a standarized low level byte code is the right way to ship cross platform applications. We already knew that. It just took a long time to go through the standarization process and be implemented by browsers. If it seems like it's not in use today, it's largely because JavaScript has momentum and the majority of web programmers don't yet know how to take advantage of wasm (or maybe don't think they need it).

necheffa 5y ago

LOLWUT? At least _try_ to understand what you are talking about before you criticize. Your rant demonstrates your complete lack of understanding on the topic. The, apparently, "hidden" knowledge is that Unix shells are designed to process text. And so you serialize complex data structures over FIFOs from child to parent. That can be as simple as comma/colon/whatever delimited bytestreams chopped up with awk or as complex as JSON and parsed with jq.

quotemstr 5y ago
Sometimes we want to write functions that have side effects. Subshell functions don't have side effects. Sometimes you don't want side effects, but sometimes you do, and it's silly to completely foreclose on even the possibility of using side effects.
- chaps 5y ago
  Isn't grep your friend in this situation? I'm really not sure why you can't have side effects in this scenario.
  - quotemstr 5y ago
    What on earth does grep have to do with side effects?
    - chaps 5y ago
      And what makes you think side effects aren't possible in the model described? Think of grep as a way to control flow. If one side effect happens, send to stdout one thing. Another side effect, send something else. Then iterate over those outputs.
  - rablackburn 5y ago
    A “side-effect” in this case is the ability for a function you call to do something “outside” of itself. For example, to change the value of a global variable. It’s generally considered best practice (in the functional programming community at least) to write “pure” functions (that is, functions without side effects) because it’s much easier to reason about what they are doing. So good news, subshells can _only_ be pure (the only way to get anything back from them is if they write some string to stdout), but sometimes you do actually want to have some side-effects (imagine a function that reads a config file and then wants to set some of the global variables to the values found there). Well, bad luck. If you use the subshell syntax you literally can’t do that.

zsz 5y ago

Sir! Yes, sir!

quotemstr 5y ago

You're right about the disadvantages of subshells, but wrong about this: > avoid bashisms What's holier-than-thou is insisting that people stick to a dialect of shell scripting that hasn't changed in decades because some people think that conforming to POSIX is its own reward even if it makes life harder and programs worse. No, thanks. I'll stick to bashisms. Things like coprocesses and filemap make certain classes of program much easier to write. And yes, shell scripts are programs, and the quickest way to turn these programs into unmaintainable balls of mud is to follow your advice to avoid normal programming best practices simply because the program you're writing happens to be a shell script.

presto8 5y ago

Here is a script that shows exporting results from a subprocess to a parent shell using a temporary file: #!/usr/bin/env bash process_line() { if [[ $* == *foo* ]]; then echo "found_foo=1" >>$IPC fi } IPC=$(mktemp) find . -maxdepth 1 -type f | while read line; do # note: this is run a subprocess process_line "$line" done source "$IPC" echo "found_foo=$found_foo"

rhizome 5y ago

If you want to program, use a programming language.

tomjakubowski 5y ago

> So you can't define an array in a subshell and then use it outside the subshell, and you can't return an array (or anything that isn't a string) from a subshell Huh. How does one return a value from a bash function? I've always only ever seen the "echo calling convention" used in scripts, or use of global variables.

kps 5y ago
Bash does have `typeset -n` (from ksh) which allows shell variables to be passed to a function by reference.
- cryptonector 5y ago
  Yes, and which gets you Tcl-ish pass by name semantics.
- tomjakubowski 5y ago
  Cool, thanks!

cryptonector 5y ago

> Getting anything out of a subshell that isn't from STDOUT is impossible. You can use I/O redirection with arbitrary file descriptor numbers, and you can let the shell pick them. So you can `mknod` a pipe, `exec {pipefd_rw}<>"$pipe_name" {pipefd_ro}<"$pipe_name" {pipefd_rw}>"$pipe_name";` and now you have three open file descriptors that you can let the sub-shell use to communicate with the parent. And so you're not limited to stdin/stdout. I do wish there was a built-in for making an anonymous pipe instead of having to make one on filesystem. (You have to open the named pipe for read-write first because otherwise you'll block, but if you don't want a read-write FD for it you can close it after opening the read and write FDs.)

foxes 5y ago

Yep bash is not a real language. If anything you are saying it should be deprecated in place for a real language. If people want to do more advanced things why does the basic language of their shell not support it? Why should you not care about the correctness of your shell scripts which can run important things? Seems a perfect place for a functional language. It’s high level, you can just declare what you want. I imagine your interpreter can add in all sorts of checks to help make your scripting more correct.

addicted 5y ago
Powershell attempts to use a more scripting friendly language in the shell. And it’s a horrible shell experience. It’s great for scripting (but not as good as other scripting languages) but it’s a terrible experience on a day to day basis, with the only salvation being the existence of 2-3 letter aliases which make things slightly more manageable. Bash is a great shell language that whose scripts are useful because it allows you to trivially convert your manual shell actions into an executable script. If you need to do anything complex (or rather, something you probably wouldn’t type into the shell manually if you had to do the thing on a one off basis), then you’re probably better off using a different scripting language. But changing BASH with scripting as a priority would probably make it a worse shell language (which is not to say there are no improvements to be made…there are).

dotancohen 5y ago

> Getting anything out of a subshell that isn't from STDOUT is impossible. Yes that is a known limitation that has horrible, horrible workarounds. Don't use them. That said, for scripts whose main input _is_ from a file or STDIN, and also whose main output _is_ to a file or STDOUT, then bash is far more often the right tool for the job than it is given credit for. Of course, I'm talking about bash and a host of other utilities that are often packaged with it such as awk, grep, sed, cut, etc. For processing text I find myself often choosing between bash and Python, and very often if I choose bash I'll feel that I've made the right choice.