5 ms·
Being a bash developer in the 21st century (2021)
- dwheeler 3y agoI have 1 key recommendation: if you're writing shell scripts, use shellcheck. It can detect many problems and hrlp you write better scripts.
- keepamovin 3y agoYes! I only wish there was a better interface. Like an editor integration. Just something where you can go through each check line 1 by 1 and consider changing or not. I tried writing this in a simple way but gave up...
- frou_dh 3y agoUse the editor's normal support for running a build/compile to run shellcheck with the `--format gcc` flag. Shellcheck will output in the traditional path:line:column:message format, which will be navigable like compiler errors.
- BozeWolf 3y agoJetbrains has shellcheck integrated in their editors:) I bet vscode can do the same.
- FredrikMeyer 3y agoEmacs' flycheck integrates perfectly with shellcheck.
- wodenokoto 3y agoI don't know which editor you are using, but the shellcheck plugin for VS Code works really well. Underlines warnings and errors, and on hover gives you a short description and a link to a long description. There's also a "fix" button, but I it only works for the simplest of cases.
- keepamovin 3y agoOh, I use vim. And I don't really know about any plugins that work. But I would try VS Code just to go through my shell files with shellcheck! :)
- marcelarie 3y agoyou can use nvim then, it has shellcheck for diagnostics and formatting, like in vscode :) here the link to the config: https://github.com/jose-elias-alvarez/null-ls.nvim/blob/main/doc/BUILTIN_CONFIG.md#diagnostic-config https://github.com/jose-elias-alvarez/null-ls.nvim/blob/main...
- keepamovin 3y agoThanks, for the link, man! Helpful. Is nvim, like, command line? I don't like to be in GUI when I'm coding. I mean I guess I could hook up my Quake style iTerm2 dropdown overlay to display a GUI window (but I think that only works on Windows with ConEmu). It's just easier for me to stay in command line, switch to GUI to eval or research...or watch videos! Haha
- llimllib 3y agonvim is a fork of vim, so yes it’s a command line editor
- keepamovin 3y agoCool! Will give a try.
- ramses0 3y agoSyntastic, I think? `~/.vim/pack/default/start/syntastic/README.markdown`
- JCWasmx86 3y agoA lot of editors have available integrations for Shellcheck. VSCode and IntelliJ have a plugin, GNOME Builder comes with the plugin included, etc.
- marcelarie 3y agoIn neovim I would recommend setting shellcheck with null.ls or using the bash language server: https://github.com/bash-lsp/bash-language-server https://github.com/bash-lsp/bash-language-server both work really well
- shepherdjerred 3y agoReading the "Technology Overview" section of the linked "How I built a modern website in 2021" [0] site is terrifying. It lists something like 15-20 technologies to build the site. I guess most of them are beneficial, but... why are some so proud of how large their dependency tree is? [0]: https://kentcdodds.com/blog/how-i-built-a-modern-website-in-2021#technology-overview https://kentcdodds.com/blog/how-i-built-a-modern-website-in-...
- MonaroVXR 3y agoI knew it, even a seprate library for tabs and more is used. It's far more than I thought.
- arcanemachiner 3y agoLOL, I remember seeing this before. It's like a piece of postmodern art.
- geocrasher 3y agoThis post inspired me to add a similar post to my own blog. Why? Today I needed to parse some JSON at the command line and do some basic search and replaces and convert it all to CSV. The API call was easy enough with wget, and I was going to use JQ to parse the JSON. But after fiddling with it for 20 minutes and not having much luck, I resorted to my favorites: Awk, sed, grep, cut. Save API call output to a couple of temp files, parse it with the aforementioned Unix staples in a for loop, And unless time than it would have taken me to properly learn JQ, I had the output and was able to ship it off to my customer. Sometimes the best tool for the job is the tool you know, love, and trust.
- oweiler 3y agochatgpt is really good for creating the correct jq queries.
- geocrasher 3y agoIt's funny that you mention that! I was discussing it with a co-worker to see if they were any better with JQ than I was. They weren't. But then I realized I probably could have used chat GPT to do it. It really is a great tool for that kind of thing.
- ykonstant 3y agoIndeed, the greatest triumph of ChatGPT is to reduce the need to discuss with co-workers! :D
- bradknowles 3y agoTry `gron`. Great for taking JSON and converting it into something that can be grep'ed, and then if you need JSON out on the other end, you can then use `gron -u` to reverse the operation. Saved my sanity more times than I can count, and allows me to avoid the Cthonic black magic known as `jq`.
- BirAdam 3y ago
- kyleyeats 3y agoThis is quaint, like an article about how bad opiates are, but it's from before fentanyl. We bash developers are in a really bad spot. If you know one of us, please check on him or her. ChatGPT has enabled our digital hoarding like never before (some of us call it a "scripts folder"-- check for files like build5.sh or *.yml for really bad cases). I don't know how to write bash. But I'm doing for loops and if statements like never before. I justify it by saying I'm only hurting myself. I've already gone through one hard drive, writing a new index.html over and over and over to the same spot, burning it into the device like the "2" in Sonic the Hedgehog 2 on a 90s CRT. I lost one of my best friends after I asked him why we don't just use string literals and yaml files for everything. Another when I told him to just put his CSS in a style tag. I can't stop. I can't go back. All I can do is stay away from Makefile people and hope it doesn't get even worse. Don't write shell scripts, kids.
- keepamovin 3y agoHahaha, lol. Where does this come from? I've certainly accelerated my Bash game since ChatGPT came to Earth. Do you know there's a syntax for defaults? ${MY_BIG_VAR:-your_tiny_default} Or that you can completely non-sensically extract the file extension with? file_extension="${1##*.}" How to pronounce that? dollar open-brace 1 pound pound star dot close-brace Reminds me of those codes on the phone system to get service functions...*69#
- blueflow 3y agoThat's called parameter expansion, see https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_02 https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
- keepamovin 3y agoAlright, when you put it like that, it does make sense. Thanks, that's a great resource! I'm gonna start using ${MY_ESSENTIAL:?"Error: need to supply an essential"}
- jrm4 3y agoThis feels like something I would write, except without feeling bad about it. Look, I'm pretty familiar with other languages, but the fundamental deal is this: I don't write programs for anyone but myself. And I've seen no reason to switch to anything else, even python. I get that things get ugly, but I absolutely prefer my own ugly to someone elses boring? I want to make things happen now.
- keepamovin 3y agoI relate! But funny about the article because I read it as the guy had an ironic tone to exaggerate the negatives, but he actually loves it. I don't think he actually feels bad about it!
- arpa 3y agoyou are absolutely right!
- ewokone 3y agoI really liked that article! I've coming to bash this year and in my company we make things work the same way (principle). Chatgpt was a great helper and instructor to me learning bash and brought me up to Speed in no time. After my first weeks I was writing bash scripts that I thought was not possible to do in a shell which wants me to 'feel home' when I open a shell - that resonated with me in that post a lot. I would be happy to find more bash script(er)s out there, to find inspiration and solutions for problems that increase my horizon of what's possible. Kudos to OP!
- ndsipa_pomu 3y agoCheck out Greg's wiki - it's a fantastic resource for BASH knowledge. https://mywiki.wooledge.org/BashFAQ https://mywiki.wooledge.org/BashFAQ Also, always validate your scripts using shellcheck (https://www.shellcheck.net/ https://www.shellcheck.net/)
- arpa 3y agoThanks mate!
- apienx 3y ago“Here are some tips on how to make your shell scripts safer. Don't.“ https://sipb.mit.edu/doc/safe-shell/ https://sipb.mit.edu/doc/safe-shell/ I say this as a diehard shell fanatic. The closer to metal, the better. ;-)
- sgarland 3y agoset -euo pipefail causes more confusion than good, IMO. Not everything has an exit (or return, more specifically) code of 0 for success, which is insane, but it’s shell. I’m good with -u on its own, but the others, meh… that’s what shellcheck is for. EDIT: Read this [0] and weep/revel in its glory, depending on your POV. [0]: https://mywiki.wooledge.org/BashPitfalls https://mywiki.wooledge.org/BashPitfalls
- throwaway46281 3y agoI agree that -o pipefail is not great since it's totally fine for pipes to break in some cases (e.g. I've occasionally seen it cause problems with pipelines like `grep | head`). Hard disagree about -e though. I get it's not perfect but it has probably saved me dozens of hours of frustration and heartbreak. If you forget to handle errors for a particular line, then the script will just keep going and that's how you accidentally delete your entire home directory. Sure you can rely on shellcheck but for me personally I don't have shellcheck available 100% of the time as I often find myself writing scripts as raw strings (e.g. ssh -c).
- ndsipa_pomu 3y agoI'm a fan of Bash3 Boilerplate (https://github.com/kvz/bash3boilerplate https://github.com/kvz/bash3boilerplate) and use a modified version of it for my shell scripts now. I like the code style and the logging, although I've amended the logging slightly to add a pipe input for long running processes that you want to see the output from before it finishes. I prefer having a stop-on-unexpected script as it makes errors far more explicit and it's not too onerous to work round the peculiarities of return codes. Greg's wiki (https://mywiki.wooledge.org/ https://mywiki.wooledge.org/) is my go to resource for looking up snippets and learning to avoid the footguns - that and shellcheck are the key to "robust" bash scripts.
- sharas- 3y agohis last name is BArysas, the site is BArysas.eu he is fond of BAsh
- keepamovin 3y agoI think British Airways is running a sophisticated orthogonal marketing campaign ... hahaha
- superkuh 3y agoBash is great generally because Bash devs write for a stable target and don't concentrate on using the latest and greatest bleeding edge features. So you can expect a bash script written today to run on a distro from 15 years ago. This is something few other programming languages have achieved (except perl). With python or rust or whatever a compiler/toolchain basically only lasts 3 months because some new feature is added and used that makes your rustc compiler from 4 months ago unable to compile it. Or with python you pretty much have to assume you system python won't work and install an entirely new toolchain using a dependency manager manager to manage your containerization and then within that (venv, pyenv, conda, etc, etc) a dependency manager (pip, PyPI, etc) to actually get the lib versions you need. Only then can you start trying to actually execute the program. Bash? Just chmod +x and run the script. It'll work no matter where you're running it. And it won't stop working.
- t43562 3y agoYou will get people who decry bashisms that pollute the bourne shell. IMO they often make things safer so I want them - like [[ ]] which works so much more expectedly than [ ]. Shells are good at starting, stopping, backgrounding processes and piping, redirecting and substituting output that they make programming languages feel very clunky imo. Plus you can now almost expect bash to always be installed on a machine somewhere.
- arpa 3y agowow, didn't expect to see my blog on HN. Thanks, I guess!