7 ms·
Shell Style Guide from Google
- pwd_mkdb 10y agoi try to write my scripts in such a way that if all newlines were lost, the script would still run. semicolons even where optional. requiring the use of bash for non-interactive use? good grief. is it possible this company has a linux bias? the usefulness of a minimal scripting shell cannot be denied. even with linux distribs that use bash, we almost always see busybox in use. busybox is more like the almquist sh than bash. with sh, bash-only features may not work. for example, shellshock did not work with almquist sh. why are bash script not given a .bash file extension to distinguish them from sh scripts (.sh extension)? .ksh extension is often used for korn shell scripts.
- chubot 10y agoGiven that every server and most desktops at Google are Linux, and that it created at least two operating systems based on Linux (Android and ChromeOS) -- yes, I think it would be safe to say there's a Linux bias :)
- coldtea 10y ago>is it possible this company has a linux bias? Why would they have any other bias? Is there any other viable alternative on the server side they should car about? Are Google realistically going to switch to it? It's not like 1990, where you coded so that your script worked with 5+ UNIX vendors. >with sh, bash-only features may not work. Sounds like a self-inflicted problem. Just arrange so bash handles your scripts (which they do). >.ksh extension is often used for korn shell scripts. A trip down memory lane...
- ymse 10y ago> Executables must start with #!/bin/bash I've started using #!/usr/bin/env bash for compatibility with non-FHS distros such as NixOS. Happy to see arrays omitted; if you find yourself in need of bash arrays you should definitely consider a proper language. I tend to stick to POSIX shell to the extent possible, but readonly variables and [[ ]] syntactic sugar makes sense when maintainability is more important than portability.
- jxy 10y agoThose are exactly two things I don't agree. > Executables must start with #!/bin/bash Either they never use *BSD, or they put their bash there. Unbelievable choice. > If you find you need to use arrays for anything more than assignment of ${PIPESTATUS}, you should use I use array all the time and never used PIPESTATUS. So what?
- ymse 10y agoFrankly, I find hard-coding #!/bin/bash sensible in a large corporate setting such as Google. It prevents PATH manipulation and they obviously have a uniform and controlled environment. Bash arrays can be convenient, but at that level of complexity Python or Ruby are much easier to read and understand (and necessarily maintain).
- riskrisk 10y ago> .. they never use *BSD .. pretty much
- asuffield 10y ago(Tedious disclaimer: my opinion only, not speaking for anybody else. I'm an SRE at Google.) We run everything on prodimage, so operating system compatibility isn't really an issue. Realistically, any time somebody sends me a code review with a shell script in it, my first comment is going to be "Why is this a shell script? Rewrite it in a supported language."
- mcsaucy 10y agoRe: arrays -- I generally agree, but they are best solution to certain otherwise simple problems, like storing command arguments. `cmd $FLAGS` is objectively more dangerous than `cmd "${FLAGS[@]}"` due to the globbing and word-splitting issues the former exposes. In my experience, devs are more likely to opt for the former when the latter is forbidden than switch languages (because that's work).
- SoapSeller 10y agoI was expecting for style guide on how to design shell user interactions. Nontheless, Shell Coding Style Guid is interesting.
- xufi 10y agoThat's a good point. I've been trying to find a good guide on make a nice MOTD (Message of the day Script) and some ASCII art since I've been playing around with my envrioment more.
- pyed 10y agois https://github.com/mvdan/sh https://github.com/mvdan/sh relevant ?
- ronjouch 10y agoTangentially, http://www.shellcheck.net/ http://www.shellcheck.net/ is an awesome static linter for shell code. Available online, as a command-line tool, but most importantly as a plugin for your favorite $EDITOR.
- careersuicide 10y agoAh! Thank you so much for posting this! Whenever I attempt to learn a new language I find that static analysis tools and linters, if available, help me get over any initial confusion about what constitutes good idiomatic code. Guides like OP help too, but nothing beats a good linter. The thing is though, I've been writing Bash shell scripts for going on 15 years now, and I still overlook little details [0] all the time. I've been mulling in the back of my head for a while a theory about the need for style guides and linters: if a language makes them a near necessity for reasons other than aesthetics then the language is probably best avoided. I'm not sure if agree totally with that. However, I think the most frustrating experiences I've had while programming are the result of languages which allow for multiple ways, in terms of syntax, of doing the same thing which behave the same most of the time. 0: https://github.com/koalaman/shellcheck/wiki/SC2086 https://github.com/koalaman/shellcheck/wiki/SC2086
- ronjouch 10y ago100% agreed! When writing bash, there are so many hoops to jump through (e.g. syntactic traps, bashisms to avoid) that people who like me who use these languages only occasionally will shoot their foot repeatedly. For this population (and I think it's big, given the glue nature of *sh), a guide won't help much because occasional practice won't give them any time to sink in; interactive linters are a godsend.
- giantninja 10y agoAre there any good example scripts that show actual examples of most of these cases? I think that would be a great supplement supplement to this guide
- ronjouch 10y agoYes but they're hidden behind the little "Play" button at the left of each point. Clicking it reveals further explanation and, often, examples. Not very discoverable; even though this feature is documented at the top right of the page, I skipped it too and only found it by accidentally clicking one of the buttons while obsessively-compulsively selecting text I was reading :D
- Razengan 10y agoThe "disclosure triangle", I believe. :)
- la6470 10y agoFuck Google I am tired of Google engineers acting as GOD and dictating the rest of the world what to do. I will damn well use ksh if I want to ...
- codemac 10y agoHard tabs for life. Frankly, wish they had chosen a better shell like es or rc for all to be written in.
- giovannibajo1 10y agoAny reason why they don't mandate something like "set -euo pipefail" at the beginning? I find it invaluable for writing and debugging scripts, and in general for avoiding weird (or dangerous) errors.
- the_common_man 10y agoGood rules overall, we follow similar guidelines. Any comments on: * readonly - describing these in library scripts is a bit dangerous since the readonly is not specific to the file. So, unless you have some namespacing pattern for globals, this will bite you * I didn't see any recommendation but always start with set -eux -o pipefail. set -e has it's own set of pitfalls but it's best to learn what the pitfalls are and work with it :-) It helps in the long run. * When using set -u, use the FOO={FOO:-} syntax to initialize defaults for all your env vars.
- codys 10y agoOn point 2: indeed, given that they are restricting themselves to bash, I would have expected a note of `set -eu -o pipefail`, at least. `-x` (tracing), isn't appropriate for many scripts, though. On point 3: `: ${FOO:-some default value}` is the typical pattern for ensuring a default value is set.
- ymse 10y agoI tend to avoid `set -e`. To borrow the Python adage: "explicit is better than implicit". In shell it translates to "if you need error handling, add it where necessary". See http://mywiki.wooledge.org/BashFAQ/105 http://mywiki.wooledge.org/BashFAQ/105 for some (most) of the pitfalls.
- d0mine 10y ago"Errors should never pass silently." is more relevant for `set -e`
- ymse 10y agoExcept they do even with `set -e`. Especially if you forget to/can't set `pipefail` (which is bash-specific). Or consider the last example from the wiki link above: set -e f() { local var=$(somecommand that fails); } f # will not exit
- LukeShu 10y ago> SUID and SGID are forbidden on shell scripts. Kinda funny, the Linux kernel just ignores SUID/SGID on script files.
- treve 10y agoMakes even more sense to forbid them then.
- Retr0spectrum 10y ago> While bash does make it difficult to run SUID, it's still possible on > some platforms which is why we're being explicit about banning it.
- superuser2 10y agoWhy can't we have setuid on shell scripts which are non-writeable? I have often wished for a facility to allow unprivileged users to execute specific predefined tasks, and writing C programs for such things would be a huge pain.
- ekimekim 10y agoThe issue is that there's an unavoidable race condition which renders SUID for interpreted (ie. #!) executables insecure. Suppose there's some SUID bash script owned by root in the system that starts with "#!/bin/bash". I (an unprivileged user) create a symlink "./foo" to "/path/to/SUID_script". I execute "./foo". The kernel follows foo, reads the SUID script's #!, and so runs "/bin/bash ./foo". Honoring the SUID, bash is run as root. Here's the race. In between the kernel doing this and bash finishing initialisation and reading its script arg, I swap out the "foo" symlink to instead point at "./my_evil_code". So bash reads "./foo" and executes the content as root. I now have arbitrary code execution as root. Are there ways this could be fixed? Possibly. But not easily. It's not enough to just say "SUID can't work through a symlink" - I could've symlinked to the SUID script's parent directory, for example. You can't say "the kernel should pass the fully resolved path to the interpreter", this will break a million scripts that rely on changing behaviour based on $0. I share your pain about wanting an easy, non-compiled means of creating a small SUID program. But I don't see any good way unless something changes. Maybe a util that takes a #! interpreted file, adds an ELF header and some machine code that execs the desired interpreter and feeds it static script content?
- gman83 10y agoIndentation Indent 2 spaces. No tabs. Use blank lines between blocks to improve readability. Indentation is two spaces. Whatever you do, don't use tabs. For existing files, stay faithful to the existing indentation. Richard Hendricks is not going to like that!
- andrepd 10y agoI don't wish to dig up a holy war, but why would you want to use spaces instead of tabs? That's what they're for, isn't it? One tab = one indentation level. No need for messing around with spaces, which serve other purposes.
- Sylos 10y agoThe usual argument is that a space is always the same width, whereas a tab can have varying widths depending on your editor. So, if you indent something with 4 spaces, then other people viewing your code will also always see 4 spaces, therefore ensuring that whatever you found to be readable indentation, does actually come out like that on the other end.
- Retr0spectrum 10y agoBut why does that matter? The reader can adjust their tab-width setting to whatever they prefer.
- scrollaway 10y agoSome people absolutely want indentation to be one specific size for everybody and don't fully understand that what might look readable for them at 2 space-widths, would be more readable for some others at 4 space-widths. Tabs are accessibility. People use spaces for the same misguided reasons they use px font sizes in css. (Rant over. Yeah, this annoys me.)
- roblabla 10y agoMy personal reason : Because it makes aligning stuff harder than it should be. I often have conditions that span multiple lines, and if they aren't aligned, it makes everything harder to read. By using spaces, I make sure the alignment is good for everybody. I'm sure there are other reasons too.
- nickysielicki 10y ago> Bash is the only shell scripting language permitted for > executables. > [...] > The only exception to this is where you're forced to by whatever > you're coding for. One example of this is Solaris SVR4 packages > which require plain Bourne shell for any scripts. > [...] > When to use Shell > > * If you're mostly calling other utilities and are doing > relatively little data manipulation, shell is an acceptable choice > for the task. > > * If performance matters, use something other than shell. > > * If you find you need to use arrays for anything more than > assignment of ${PIPESTATUS}, you should use Python. > > * If you are writing a script that is more than 100 lines long, > you should probably be writing it in Python instead. Bear in mind > that scripts grow. > > * Rewrite your script in another language early to avoid a > time-consuming rewrite at a later date. Limiting the domain of shell scripts is the best advice in here. Shell scripts are terrible to maintain, and you should use perl or python if it's complicated. But limiting the domain of shell scripts also probably means that you're not going to be using any advanced features of bash, so you should write it in sh instead. It's undoubtedly more portable, and it also serves to enforce the idea that you shouldn't write complicated shell scripts. I'll take a leap and say it outright, if you're using a feature of bash that isn't in bourne shell, you shouldn't be writing it in either of them. The internet is littered with bash scripts that are compatible with bourne shell, but the shebang says '#!/bin/bash'. Well some systems don't have bash! But everyone has bourne shell. I'm disappointed by their choice of bash and how this guide might reinforce that behaviour, but other than that, the actual style guidelines are great. [Edit: rewording]
- Zardoz84 10y ago> * If you are writing a script that is more than 100 lines long, > you should probably be writing it in Python instead. Bear in mind > that scripts grow. > > * Rewrite your script in another language early to avoid a > time-consuming rewrite at a later date. Hehe. I did on DLang with a script that we have to update some instances of our application on develop machines. Our old bash scripts was becoming bigger and can't handle any more some few cases.
- 10y ago
- hobarrera 10y ago> Bash is the only shell scripting language permitted for executables. Why? That seems completely arbitrary. I already lost interest in following this in the first line. > Executables should have no extension (strongly preferred) or a .sh extension. Libraries must have a .sh extension and should not be executable. So, not only are they bash-only, but you add an extension that makes one assume the contrary. UGH!
- ams6110 10y agoThe answer to your question is obvious if you've ever worked in a large development organization.
- hobarrera 10y agoYou could have bothered sharing it, if it's so obvious.
- havetocharge 10y agoYou have much to learn. You should be reading more of this stuff, not less.
- deleted 10y ago[deleted]
- paule89 10y agoGoogle seems to prefer spaces over tabs
- ams6110 10y agoI prefer spaces. Tabs can render differently in different editors, making alignment harder. Always irks me when I come across some config file format that only accepts tab delimeters.
- ekimekim 10y agoWhich is why I take (well, prefer) a third option: Tabs for indentation. Spaces for alignment. The idea is you should be able to set any tabstop you like and the code will still align correctly. For example: ^Ia_really_long_method(a_very_long_argument, ^I the_second_argument)
- sorenjan 10y agoThis is the most logical option. Tabs to add an indentation level, the width of that is unimportant and can be configured by each user. Spaces to align, since they're like any other character in a mono spaced font thereby perfectly matching the line above.
- rifung 10y agoGolang uses tabs and not spaces though.. I actually thought everyone used spaces instead of tabs till I had to use it.
- riskrisk 10y agogolang also had/has a style guide and a gofmt tool that would indent/align things the same way from the beginning.
- e40 10y agoAnd 2 spaces is way too little. 4 is the sweet spot. Reason: with 2 the visual lines at different indent levels are just too close, in many fonts/sizes.
- darekdk 10y agoXML. Woah.
- jdgiese 10y agoLove this! If you think you are a BASH pro, check out your knowledge with some challenge problems: http://innolitics.com/10x/advanced-bash-exercises/ http://innolitics.com/10x/advanced-bash-exercises/ Also has some other useful BASH links and comments. Here is a quote: > BASH is the most widely-used and widely-supported shell for Linux. There are other shells that are better than BASH in various ways, but we feel that none of these other shells are better enough to warrant replacing BASH as the de-facto standard when writing shell scripts. BASH is installed by default on almost all Unix-based operating systems, and the majority of the world’s shell scripts are written in BASH. For this reason, we suggest that all of our developers learn BASH. > BASH scripts are a domain-specific programming language that is well-suited to managing processes and files. That being said, the large number of special characters appropriated for process management, its text expansions, and its unusual syntax make BASH poorly-suited for general purpose programming. Accordingly, we think that BASH should only be used for scripts that are predominantly concerned with processes and files.
- sytse 10y agoInspired by this we made a guide to shell out from ruby in a safe way: https://github.com/gitlabhq/gitlabhq/blob/master/doc/development/shell_commands.md https://github.com/gitlabhq/gitlabhq/blob/master/doc/develop... (on mobile and only seeing the GitHub link in Google)