8 ms·

I am a bash expert. This paper reads like it was written by clueless academics with no real world experience. Multicore bash already exists and is seldom used

I am a bash expert. This paper reads like it was written by clueless academics with no real world experience. Multicore bash already exists and is seldom used, through the "parallel" cmd. Other points address no particular issue, and would actually make the language more painful to use. Bash has its limits, which pushes you to use a full fledged language if you attempt to walk past them. It's a good system.

thanatos519 5y ago
I'm also a bash expert and I agree! I wrote 75 lines of bash to do failure-resistant streaming parallel computation on heterogeneous distributed systems. "parallel" was very close to doing what I want, but it wasn't particularly hard to do it myself without it. I fire it up, peg every core on as many spot instances as AWS will give me, then shut them all down. I regularly pass data structures through pipes, and "jq" does everything I have needed so far. The tools can always get a bit better, but I don't need my duct tape to be any smarter, KTHX.
- arpa 5y ago
  what makes one a bash expert? genuinely curious if i could call myself one.
  - deleted 5y ago
    [deleted]
  - Bluecobra 5y ago
    What makes me an expert is the amount of computer books I have near my desk. Sort of like how the enchantment table works in Minecraft. I am also a bash expert because I own an O’Reilly book on the subject I bought at a Borders 20 years ago at retail price. :)
    - arpa 5y ago
      I am a bash expert as well then, and probably the only person insane enough to have a web framework, complete with http server written in bash.
      - LilBytes 5y ago
        I'd say you were both masochistic and way beyond just being an expert in that case. Bash Technical Fellow?
  - thanatos519 5y ago
    10 years ago my colleagues were astounded by my airtight bash scripts which formed the backbone of our deployment pipeline. I've come a long way since then, and I still learn new things about it regularly, so I guess I'm a bash expert.
- laumars 5y ago
  I’m also a bash expert but I disagree with your point. Bash was good enough when being a sysadmin meant your structured data was largely just flat lists of plain text. However the industry moved away from that model years ago. And these days it’s more pronounced than ever with myself spending more time in YAML, JSON and tables thanks to modern DevOps tooling than I have done in the 2 decades of experience I had prior. Sure, jq is amazing. But if I’m having to reach for jq in nearly every pipeline then one has to question why the shell can’t just understand JSON natively itself. And why should I have to learn a different tool with their own formatting language for working with YAML, CSV, XML, S-expressions and so on. Why can’t I use the same shell primitives regardless of the underlying data format is? This is why I switched away from bash. I’m enough of an expert in bash to use it for DevOps everyday but I’m also pragmatic enough to realise that other shells actually do offer quality of life improvements. This doesn’t mean those shells have to be fully fledged programming languages - I’m really not suggesting that we should be using shell scripting for anything complicated. The problem is the stuff that isn’t complicated has evolved to become complicated in bash because the nature of our work has shifted away from what bash did well. But each to their own. I wouldn’t want to impose my workflow on others anymore than I’d want others to lecture me about theirs. I just wanted to offer a counterpoint from someone who is also a seasoned sysadmin (and more recently “DevOps”).
  - thanatos519 5y ago
    I see your point, and you're right - it is a matter of personal taste. That's just where I draw the line as to what the shell should understand. I'm mostly in K8s these days where JSON and YAML are interchangeable so I just need jq (or yq, which is just a jq wrapper). I'm curious, though, what other shells offer 'quality of life improvements' ... and can I convince my colleagues to install it everywhere?
    - laumars 5y ago
      > I'm curious, though, what other shells offer 'quality of life improvements' To be honest most alt of them do. Bash has enough footguns to incapacitate a small army. Seasoned users like ourselves know how to mitigate them but using a shell that doesn’t require a conscious effort to support spaces in file names and other really basic things is one less piece of mental overhead I need to consider when working. > and can I convince my colleagues to install it everywhere? I’m not sure what you mean here. Are you trying to convince your colleagues to switch so you all use the same shell or saying you still spend a lot of time in remote SSH sessions? Either way, it sounds like you’re probably best sticking with Bash if those are your requirements. My remote workflow is largely API driven (serverless) and the few times I do need to SSH into a remote instance it’s usually just to perform basic tasks. The real meat of my console work is done locally these days.
futharkshill 5y ago
Could you tell us, why would you choose to use Bash compared to other programming languages if you are making complex programs?
- louiskottmann 5y ago
  Like I said in the last paragraph: I would not. If the bash program evolves to span multiple files and lots of lines, I usually switch to ansible/ruby/clojure
- dfinninger 5y ago
  I think your parent poster is implying that they don’t use Bash for complex programs.
  - thanatos519 5y ago
    I believe that was the implication, too. One's level of expertise will determine what is "too complex for $SHELL". When that point is reached, it's time to switch to something else, not time to add more features to $SHELL. (If, when something becomes too complex for $SHELL, you switch to assembly, you might be too much of a $SHELL expert!)
- marklgr 5y ago
  Not OP, but I would say that we, regular Bash users, don't really write complex programs in Bash, even though it depends on what you call "complex". Some of our programs can be pretty long, and we can do more within Bash than some programmers know (eg. the arrays/associative arrays features, the vast parameter expansion options etc.), so no, we aren't bound to some 100-line-max rule, if that's what you had in mind. Bash (and other shells) are very good at working with other programs and using the filesystem. It's true you have to know many idioms, but once you're there, you can be quite productive for these kinds of tasks.
  - rhn_mk1 5y ago
    For me, anything that I couldn't write without a guide is too complex. Like a function. Or a conditional. Subjectivity of what is complex is the problem here, which means that regular Bash users regularly write programs that are complex to me. And I'd much rather use something with less footguns, even if the good parts of the shell become less easy.
IshKebab 5y ago
The fact that you can cobble together some parallel execution is missing the point. Bash is an error-prone insane mess. It's not "a good system". Frankly it's so obviously terrible that I wouldn't trust the opinion of anyone that calls themselves a "bash expert"! It's like being a punch card expert. Actually that's unfair to punch cards. They were well designed at the time. Bash was born deformed.
deleted 5y ago
[deleted]

thanatos519 5y ago

I'm also a bash expert and I agree! I wrote 75 lines of bash to do failure-resistant streaming parallel computation on heterogeneous distributed systems. "parallel" was very close to doing what I want, but it wasn't particularly hard to do it myself without it. I fire it up, peg every core on as many spot instances as AWS will give me, then shut them all down. I regularly pass data structures through pipes, and "jq" does everything I have needed so far. The tools can always get a bit better, but I don't need my duct tape to be any smarter, KTHX.

arpa 5y ago
what makes one a bash expert? genuinely curious if i could call myself one.
- deleted 5y ago
  [deleted]
- Bluecobra 5y ago
  What makes me an expert is the amount of computer books I have near my desk. Sort of like how the enchantment table works in Minecraft. I am also a bash expert because I own an O’Reilly book on the subject I bought at a Borders 20 years ago at retail price. :)
  - arpa 5y ago
    I am a bash expert as well then, and probably the only person insane enough to have a web framework, complete with http server written in bash.
    - LilBytes 5y ago
      I'd say you were both masochistic and way beyond just being an expert in that case. Bash Technical Fellow?
- thanatos519 5y ago
  10 years ago my colleagues were astounded by my airtight bash scripts which formed the backbone of our deployment pipeline. I've come a long way since then, and I still learn new things about it regularly, so I guess I'm a bash expert.
laumars 5y ago
I’m also a bash expert but I disagree with your point. Bash was good enough when being a sysadmin meant your structured data was largely just flat lists of plain text. However the industry moved away from that model years ago. And these days it’s more pronounced than ever with myself spending more time in YAML, JSON and tables thanks to modern DevOps tooling than I have done in the 2 decades of experience I had prior. Sure, jq is amazing. But if I’m having to reach for jq in nearly every pipeline then one has to question why the shell can’t just understand JSON natively itself. And why should I have to learn a different tool with their own formatting language for working with YAML, CSV, XML, S-expressions and so on. Why can’t I use the same shell primitives regardless of the underlying data format is? This is why I switched away from bash. I’m enough of an expert in bash to use it for DevOps everyday but I’m also pragmatic enough to realise that other shells actually do offer quality of life improvements. This doesn’t mean those shells have to be fully fledged programming languages - I’m really not suggesting that we should be using shell scripting for anything complicated. The problem is the stuff that isn’t complicated has evolved to become complicated in bash because the nature of our work has shifted away from what bash did well. But each to their own. I wouldn’t want to impose my workflow on others anymore than I’d want others to lecture me about theirs. I just wanted to offer a counterpoint from someone who is also a seasoned sysadmin (and more recently “DevOps”).
- thanatos519 5y ago
  I see your point, and you're right - it is a matter of personal taste. That's just where I draw the line as to what the shell should understand. I'm mostly in K8s these days where JSON and YAML are interchangeable so I just need jq (or yq, which is just a jq wrapper). I'm curious, though, what other shells offer 'quality of life improvements' ... and can I convince my colleagues to install it everywhere?
  - laumars 5y ago
    > I'm curious, though, what other shells offer 'quality of life improvements' To be honest most alt of them do. Bash has enough footguns to incapacitate a small army. Seasoned users like ourselves know how to mitigate them but using a shell that doesn’t require a conscious effort to support spaces in file names and other really basic things is one less piece of mental overhead I need to consider when working. > and can I convince my colleagues to install it everywhere? I’m not sure what you mean here. Are you trying to convince your colleagues to switch so you all use the same shell or saying you still spend a lot of time in remote SSH sessions? Either way, it sounds like you’re probably best sticking with Bash if those are your requirements. My remote workflow is largely API driven (serverless) and the few times I do need to SSH into a remote instance it’s usually just to perform basic tasks. The real meat of my console work is done locally these days.

futharkshill 5y ago

Could you tell us, why would you choose to use Bash compared to other programming languages if you are making complex programs?

louiskottmann 5y ago
Like I said in the last paragraph: I would not. If the bash program evolves to span multiple files and lots of lines, I usually switch to ansible/ruby/clojure
dfinninger 5y ago
I think your parent poster is implying that they don’t use Bash for complex programs.
- thanatos519 5y ago
  I believe that was the implication, too. One's level of expertise will determine what is "too complex for $SHELL". When that point is reached, it's time to switch to something else, not time to add more features to $SHELL. (If, when something becomes too complex for $SHELL, you switch to assembly, you might be too much of a $SHELL expert!)
marklgr 5y ago
Not OP, but I would say that we, regular Bash users, don't really write complex programs in Bash, even though it depends on what you call "complex". Some of our programs can be pretty long, and we can do more within Bash than some programmers know (eg. the arrays/associative arrays features, the vast parameter expansion options etc.), so no, we aren't bound to some 100-line-max rule, if that's what you had in mind. Bash (and other shells) are very good at working with other programs and using the filesystem. It's true you have to know many idioms, but once you're there, you can be quite productive for these kinds of tasks.
- rhn_mk1 5y ago
  For me, anything that I couldn't write without a guide is too complex. Like a function. Or a conditional. Subjectivity of what is complex is the problem here, which means that regular Bash users regularly write programs that are complex to me. And I'd much rather use something with less footguns, even if the good parts of the shell become less easy.

IshKebab 5y ago

The fact that you can cobble together some parallel execution is missing the point. Bash is an error-prone insane mess. It's not "a good system". Frankly it's so obviously terrible that I wouldn't trust the opinion of anyone that calls themselves a "bash expert"! It's like being a punch card expert. Actually that's unfair to punch cards. They were well designed at the time. Bash was born deformed.

deleted 5y ago

[deleted]