9 ms·
ShellCheck: A static analysis tool for shell scripts
- DiabloD3 6y agoI use ShellCheck, its a great linting tool. All the vim linting frontends support it.
- ducktective 6y agoyeah, for integration with coc-nvim, take a look at this github comment: https://github.com/meatwallace/dotfiles/issues/129#issuecomment-560035270 https://github.com/meatwallace/dotfiles/issues/129#issuecomm... There is also a LSP at : https://github.com/bash-lsp/bash-language-server https://github.com/bash-lsp/bash-language-server
- ufo 6y agoOne of my favorite things about Shellcheck is that the error messages come with a link to a wiki page about the problem. Does anyone know other programs that have a similar approach to error messages? https://github.com/koalaman/shellcheck/wiki/Checks https://github.com/koalaman/shellcheck/wiki/Checks
- varbhat 6y agoRust Compiler Error Index https://doc.rust-lang.org/error-index.html https://doc.rust-lang.org/error-index.html
- mitmaro 6y agoRelated, Clippy, a linter for Rust also does the same and links to https://rust-lang.github.io/rust-clippy/master/index.html https://rust-lang.github.io/rust-clippy/master/index.html .
- DiabloD3 6y agoYeah, C# compilers do. Easily Googleable error code to further explain it.
- oblio 6y agoI think it's a Microsoft thing. A ton of their stuff has unique error codes just for this purpose.
- trinix912 6y agoRemember back when OS/2 had a special error code for every error message?
- SSLy 6y agoCodes like that are a part of IBM culture that spilled out to other companies.
- llarsson 6y agoSure beats the General Protection Fault we were so accustomed to back in the day. :)
- MichaelMoser123 6y agothey had the MSDN cd (microsoft developer network) for documentation in these days; that cd had a lot of thorough documentation for apis and developer tools.
- _pmf_ 6y agoAll heavily internationalized software should have error codes.
- oblio 6y agoPHP was just ahead of the T_PAAMAYIM_NEKUDOTAYIM curve.
- KronisLV 6y agoI recall working with AngularJS, which output error messages that had links, which opened the documentation with bits from your stack trace filled in. It was a while ago and seemed a bit gimmicky, but i really enjoyed the ability to see what could be done about an error by opening a web page (without having to manually dig through StackOverflow or other sites). Now, if we'd get a development framework that'd couple documentation like that, with user generated comments (like PHP has, for example see the bottom of https://www.php.net/manual/en/function.strcmp.php https://www.php.net/manual/en/function.strcmp.php ), then i think the developer experience would improve a bunch! Who knows, maybe even allow users to contribute possible fixes and allow those as autocomplete solutions and you've just improved on what tools like Codota do a whole bunch!
- thepratt 6y agoHadolint is another. It's built atop shellcheck. https://github.com/hadolint/hadolint https://github.com/hadolint/hadolint
- tyingq 6y agofind-sec-bugs does that. It's used by, for example, SonarQube. See https://github.com/find-sec-bugs/find-sec-bugs/blob/master/findsecbugs-plugin/src/main/resources/metadata/messages.xml https://github.com/find-sec-bugs/find-sec-bugs/blob/master/f...
- rakingleaves 6y agoError Prone does this: https://errorprone.info/bugpatterns https://errorprone.info/bugpatterns
- imoverclocked 6y agoErrorprone (java compiler plug-in) has this as does NullAway (an errorprone plug-in) I agree that it’s really handy, especially when implementing the check in a CI pipeline with lots of developers.
- monoideism 6y agoMany of the JS linters do as well (eslint, etc).
- human_error 6y agoSQLAlchemy also does something similar. It points to relevant SQLAlchemy page for more information about the error.
- darwingr 6y agoDart's static analyzer & lint rules: https://dart-lang.github.io/linter/lints/ https://dart-lang.github.io/linter/lints/ Each rule has its own url. Tool therefore easily gives you a link.
- ptman 6y agoIf you're building a REST API, make sure to check out RFC7807 ( https://tools.ietf.org/html/rfc7807 https://tools.ietf.org/html/rfc7807 ) , which recommends doing exactly this. Giving each error a type, which is a resolvable URI.
- agapon 6y agozpool status
- account42 6y agoNewer versions of GCC have clickable (using terminal escape sequences) warning flags in the output that link to the corresponding documentation.
- jidiculous 6y agoStaticcheck for Go kind of does something similar in that it outputs Googleable error codes, but doesn't actually print a link to the pages. It's also included as part of golangci-lint. https://staticcheck.io/docs/checks https://staticcheck.io/docs/checks
- knoebber 6y agoShellCheck made me confident enough to write more complex bash scripts. Highly recommend! Works great with emacs + flycheck.
- ollran 6y agoIt is also a quite nice tool for checking the POSIX compatibility of shell scripts. I find it very useful when working on Solaris or some other UNIX that does not use Bash by default.
- lipanski 6y agoSame here, I basically learned how to write POSIX-compliant Shell with the help of ShellCheck. Shell is not a difficult language to learn but it definitely has its oddities and the different flavours (Bash etc.) only add to the confusion. What people don't always expect is the immense portability that POSIX-compliant Shell offers you. This thing runs on pretty much everything.
- pabs3 6y agoAnother tool for that is checkbashisms from the Debian devscripts package: https://manpages.debian.org/checkbashisms https://manpages.debian.org/checkbashisms
- agumonkey 6y agohaskell forbidding people to leave fragile bash to learn babashka !
- thitcanh 6y agoI love linters and formatters. Often languages carry baggage and other “bad parts” that linters help you avoid. For someone who’s not used to them they might feel annoying (e.g. why should I wrap every $() in quotes?! Ugh) but once you accept them your code will be a million times better and safer.
- omn1 6y agoSame. Few years ago I started a little list of linters. Just kept adding tools and the list is huge now thanks to lots of contributors. https://github.com/analysis-tools-dev/static-analysis https://github.com/analysis-tools-dev/static-analysis Still love adding new linters there when I cross their path. :)
- MichaelMoser123 6y agothanks! you also have https://github.com/analysis-tools-dev/dynamic-analysis https://github.com/analysis-tools-dev/dynamic-analysis for stuff like valgrind; A star from me for both of them.
- koyote 6y agoI believe they are near-essential for properly learning a language. I can credit Resharper for a large part of my proficiency in C# for example.
- parhamn 6y agoNot near a terminal right now but it would be fun to run my shell history through this to see if I learn anything. Anyone try it?
- jkingsman 6y agoUnless you're composing very rich commands, likely not much will show up. It exceeds at finding things like interpolation gotchas, bad loop control, etc. Much more structural than just one off commands (though would likely bug about a bunch of useless `cat`s when you can use redirect operators instead (i.e. `cat foo.txt | grep` is an antipattern according to shellcheck)
- HenrikB 6y agoshellcheck-repl: Validation of Shell Commands Before Evaluation https://github.com/HenrikBengtsson/shellcheck-repl https://github.com/HenrikBengtsson/shellcheck-repl This tool validates your commands at the Bash prompt using ShellCheck and refuses to evaluate them if there's a mistake. It ignores a set of rules that doesn't play well with oneliners. (Disclaimer: I'm one of the authors)
- devy 6y agoThis has been brought up ten times since 2014 when it first appears on HN :) Click on the site link[1] in parenthesis to the right of the title, you will see. [1]: https://news.ycombinator.com/from?site=github.com/koalaman https://news.ycombinator.com/from?site=github.com/koalaman
- llarsson 6y agoIt is actually good enough to be here that often. More often, even. It's like in this xkcd. It's always new to somebody. https://xkcd.com/1053/ https://xkcd.com/1053/
- abathur 6y agoAnd 23 more for https://news.ycombinator.com/from?site=shellcheck.net https://news.ycombinator.com/from?site=shellcheck.net :)
- anaganisk 6y agoNot once did I see it till now, its not like people are here 365 days or new people dont join.
- __bjoernd 6y agoGreat tool that everyone should be using for shell scripts! Then again, I've learnt more than once that you start with "I'm just capturing a few commands in a script" and the next thing you see is a mess of special characters, crude syntax and nasty error handling. Don't be like me...
- cylon13 6y agoI find it helps if you have a hard rewrite rule. I really love the convenience of shell scripts for small scripts, but as soon as I have one that crosses 100 lines I rewrite it in python. The python version is a bit less convenient for spawning processes and piping them together, but it's so much more maintainable that you can scale it up to quite a bit larger without as much worry. 100 lines is a nice size where it doesn't take very long to rewrite, so even though I figure I could get away with a longer shell script without it getting too bad, I'm definitely not going to want to rewrite it once it gets big enough to be a problem.
- atleta 6y agoInvoke [1] can make it a bit more convenient. [1] https://www.pyinvoke.org/ https://www.pyinvoke.org/
- kawsper 6y agoI am you, except my go-to is Ruby.
- qbasic_forever 6y agoAnd then you get to google scale and realize using python for every config, like your entire build system, leads to a huge mess of completely non-deterministic build scripts that do wacky things like make HTTP calls to prod systems, depend on all kinds of global state stores, etc. So you build a new version of python that strips out all of the non-determinism and move your config to that (see starlark: https://docs.bazel.build/versions/master/skylark/language.html https://docs.bazel.build/versions/master/skylark/language.ht... ).
- blabitty 6y ago
- acdha 6y agoThis is one of my favorite tips for almost effortlessly improving devops productivity (along with using https://github.com/mvdan/sh https://github.com/mvdan/sh for auto-formatting) — by now shellcheck has good editor support in e.g. VSCode and I have a standard pre-commit.com hook for all of my projects. I've seen so many long iterations where people flail at a complex shell script for something like a cron job which was immediately flagged by shellcheck. I generally recommend rewriting in Python but this is a good less invasive step. One thing which might be worth considering is adding this to your personal ~/.shellcheckrc to make it more pedantic: enable=all
- ptman 6y agoI went looking through the issue tracker to see if there's an issue about supporting $XDG_CONFIG_HOME and was pleasantly surprised to find out that it seems to already work. So ~/.config/shellcheckrc works just as well as ~/.shellcheckrc
- exdsq 6y agoAlongside ShellCheck, I also use shUnit2 as my unit testing framework. These two combined have let me write some pretty long (~1k loc) bash programs that work. Yes, you should test your bash. https://github.com/kward/shunit2 https://github.com/kward/shunit2
- ducktective 6y agoThere are quite a few unit-testing frameworks for bash. I wonder, have you reviewed and compared them and came to `shunit2`?
- exdsq 6y agoI tried one or two but decided on shUnit2 because it seemed to be the most friendly and familiar. If you have suggestions of alternatives I'd love to hear them!
- hivacruz 6y agoTo use in combination with shfmt : https://github.com/mvdan/sh https://github.com/mvdan/sh
- qbasic_forever 6y agoI really like task (http://taskfile.dev/ http://taskfile.dev/) as a more usable layer on top of mydan/sh. Task is basically make but with a yaml syntax and built-in posix shell interpreter all in a single statically linked executable. It makes building something even on Windows a total breeze.
- ghostwriter 6y agoA great highlight of how great Haskell parser combinators can be in the domain of linters.
- tyingq 6y agoI haven't used it, but I have seen something similar called shellharden. It pitches it's advantage as being able to auto apply the changes. https://github.com/anordal/shellharden https://github.com/anordal/shellharden
- dang 6y agoIf curious, the interesting past threads seem to be: Lessons learned from writing ShellCheck - https://news.ycombinator.com/item?id=22279585 https://news.ycombinator.com/item?id=22279585 - Feb 2020 (46 comments) Shellcheck: a static analysis tool for shell scripts - https://news.ycombinator.com/item?id=9001931 https://news.ycombinator.com/item?id=9001931 - Feb 2015 (46 comments) ShellCheck: a static analysis and linting tool for sh/bash scripts - https://news.ycombinator.com/item?id=8777705 https://news.ycombinator.com/item?id=8777705 - Dec 2014 (20 comments) ShellCheck – Online shell script analyzer - https://news.ycombinator.com/item?id=8182745 https://news.ycombinator.com/item?id=8182745 - Aug 2014 (14 comments)
- saagarjha 6y agoI much prefer this to the format you were using several months ago :)
- pvinis 6y agoI just tried to use it this morning. I had an if with [[ ]] and one with [ ] so I hoped for advice. No errors or warnings. Then I didn't bother using it anymore. Furthermore, I switched the script to use deno!
- miduil 6y agoshellcheck won't force you to prefer one of the two variants, hence the difference is relatively subtle to a degree that even documentation is not superhelpful either: https://www.oilshell.org/blog/2016/10/12.html https://www.oilshell.org/blog/2016/10/12.html shellcheck will show you many more mistakes that are much more painful your example and also very common. Deno is definitively an interesting language, but Shell scripting is the gateway to bootstrap so many of your beloved programming languages. If your environment allows it always prefer a proper programming language over shell scripting. Shell scripts are meant for Unix operating system operations, setting few variables, combining certain data-sources, doing filesystem operations, ...
- maddyboo 6y agoA useful thing with conditionals is shellcheck will warn you not to write [ foo -eq 1 && bar -eq 2 ], but to write [ foo -eq 1 ] && [ bar -eq 2 ] or [[ foo -eq 1 && bar -eq 2 ]]. I do think it would be nice if shellcheck was more opinionated, or if it was possible to create rulesets that have stronger opinions. For example, I would personally like to enforce the use of double bracket conditionals in my scripts, but I know that others prefer the single bracket variety.
- ratsmack 6y agoWhat would be better is that undesirable syntax support is aggressively deprecated and ultimately removed.
- DrewRWx 6y agoWhich shebang was your script using? If it was #!/usr/bin/env sh then shellheck would report SC3010: In POSIX sh, [[ ]] is undefined. .
- dj_gitmo 6y agoThis is one of the best pieces of software. It's really changed the game and I don't think it would be easy to safely use shell in large teams without it.
- benlwalker 6y agoSince the project I work on (https://spdk.io https://spdk.io) largely produces a set of executables as output, it was most natural to write the tests in bash. There's one top level bash script that kicks off the full suite of tests and thousands and thousands of lines of tests all written as bash scripts stringing together calls to these executables. One of these tests is to run shellcheck against all of the scripts in the repo. We don't allow any modifications to scripts without shellcheck giving them the green light now. The quality of our tests has increased dramatically since this was instituted - it's a really great tool.
- afrodc_ 6y agoCan you expand on why y'all thought Bash was a natural fit to kick off all your tests?
- zimmund 6y agoAnd talking about tests... I recommend [bats](https://github.com/bats-core/bats-core https://github.com/bats-core/bats-core) for testing! I'm slowly adding tests to my dotfiles using this.
- justToChimeIn 6y agoCan confirm, ShellCheck is awesome.
- pabs3 6y agoSome other shell script check and formatting tools: All shells take the -n option to perform a basic syntax check. Debian has a script for checking scripts for bashisms: https://manpages.debian.org/checkbashisms https://manpages.debian.org/checkbashisms bashate is a automated style checker for bash (similar to pep8 for python): https://opendev.org/openstack/bashate https://opendev.org/openstack/bashate lintshell is an early prototype of a shell linter based on the Morbig trustworthy static parser for POSIX shell, based on the Why3 platform for deductive program verification. https://www.irif.fr/~treinen/colis/ https://www.irif.fr/~treinen/colis/ https://github.com/colis-anr/morbig https://github.com/colis-anr/morbig https://github.com/colis-anr/lintshell https://github.com/colis-anr/lintshell Shellharden helps rewrite scripts for ShellCheck compliance: https://github.com/anordal/shellharden/ https://github.com/anordal/shellharden/ shfmt is gofmt for shell: https://github.com/mvdan/sh https://github.com/mvdan/sh ShellScriptFormatter is another formatter: https://github.com/osalvador/ShellScriptFormatter https://github.com/osalvador/ShellScriptFormatter Some of these tools and other tools can be automatically run by check-all-the-things: https://github.com/collab-qa/check-all-the-things/ https://github.com/collab-qa/check-all-the-things/
- ducktective 6y agoQuality links there. Thanks!
- scns 6y agoThere is a plugin for intellij based IDEs. The first thing i install on a new installation alongside IdeaVim (which is the best Vim emulation IMHO)