19 ms·
Bash 5.0 released
- giancarlostoro 8y agoAny recommended reading for Bash? I'm somewhat new to it and it's interesting ways of getting things done. I've used it minimally in the past, but have found myself writing a 100> LOC script, which I can't help but feel I'm likely over-complicating certain bits and pieces.
- gravypod 8y agoI've recently picked up a subscription to Destroy All Software and he has a few really good demos of how to architect simple and powerful shell scripts using bash. I'd really recommend it for anyone who doesn't need help with the semantics and instead needs help with the organization of a script.
- giancarlostoro 8y agoI've got a subscription too, watching him go in some of the videos as he casually does Bash blew my mind, which is another reason I ask this question. I'm trying to at least watch all his videos, I intend to work through them after watching them all at least once, and then I intend to rewrite some of the solutions (the Ruby ones) in Python and Go just to be sure I understand the concepts more, and I'm not just copying and pasting.
- abryzak 8y agoThe man pages for bash are fairly comprehensive and I'd definitely recommend referencing them while writing scripts: https://linux.die.net/man/1/bash https://linux.die.net/man/1/bash
- mishac 8y agoThe full manual is here. https://www.gnu.org/software/bash/manual/bashref.html https://www.gnu.org/software/bash/manual/bashref.html GNU projects tend have info pages that are a lot more complete and thorough then their man pages, so that'd be a good place to check too.
- chrisseaton 8y ago> GNU projects tend have info pages that are a lot more complete and thorough then their man pages Why is that? Why don’t they build both from a single source?
- int_19h 8y agoBecause they're structured differently. A man page for an app is just that, a page, with some formatting and sections. Texinfo is more like your typical website - lots of small cross-linked pages with a common index. Consequently, a man page is usually an extended take on --help, while info pages are more like product manuals. This is the theory. The practice is that GNU mandates Texinfo for its projects, and because documentation tends to be the weak spot for all open source projects, manpages end up the most neglected as a result, which can be quite annoying. Especially since pretty much nobody else uses Texinfo - man pages are good enough for most console apps, and those that need more detailed documentation use Docbook, Markdown etc.
- pbhjpbhj 8y agoThey could easily just serialise the info pages under page headings and have that as an additional manpage.
- SllX 8y agoHistorical difference in style and preferences. GNU’s Not UNIX and all that Jazz, so they have different conventions, some of which would not translate to a man page that well. Plus Texinfo is a tool for generating multiple output formats from a single source. Typically (unless this changed at some point) the info pages are the canonical reference for GNU projects and the man pages are to accommodate Unix hackers.[1] [1] https://www.gnu.org/prep/standards/standards.html#Man-Pages https://www.gnu.org/prep/standards/standards.html#Man-Pages
- JeremyBanks 8y agoIt can't be done. If you want to write reliable code, and actually notice all of the possible error conditions instead of silently ignoring them, your code needs to get more verbose and complicated than it would be to just use a more capable tool like Python or Node, and it still won't be as reliable. If you have more logic than a couple of string comparisons, Bash is not the right tool for the job.
- mohaine 8y agoAs they say, bash is just good enough not to get replaced.
- nwatson 8y agoI recommend Greg's Bash Wiki ... https://mywiki.wooledge.org/BashGuide https://mywiki.wooledge.org/BashGuide. See general notes, then at the bottom of the page are many links to additional considerations. Like others say, "bash" is a hard tool to get right (and I'm not saying I do it right either, necessarily, but Greg's Wiki was real helpful!). I'm building a hybrid bash/python3 environment now (something I'll hopefully open-source at some point), and bash is just the "glue" to get things set up so most aspects of development can funnel through to python3 + other tools. But ... things that make bash real useful: * it's available everywhere (even in Windows with Ubuntu-18.04/WSL subsystem) * it can bootstrap everything else you need * it can wrap, in bash functions, aliases, and "variables" (parameters), the real functionality you want to expose ... the guts can be written in python3 or other tools Without a good bash bootstrap script you end up writing 10 pages of arcane directions for multiple platforms telling people to download 10 packages and 4 pieces of software per platform, and nobody will have consistent reproducible environments. EDIT: I think there's a revised version of Greg's Bash Wiki in the works.
- AnIdiotOnTheNet 8y agoOr you just bundle your application and its dependencies into a single folder for each OS and distribute it.
- dorfsmay 8y ago
- saagarjha 8y ago> have found myself writing a 100> LOC script Often, this is a good sign that you might want to switch to another scripting language.
- kawsper 8y agoThat's true, so many times I have used shellcheck and started to improve a bash script, only to realize my time was better spent rewriting the script in Ruby.
- int_19h 8y agoTake a look at this, as well: https://amoffat.github.io/sh/ https://amoffat.github.io/sh/
- TheGrassyKnoll 8y agoSee also: http://www.pyinvoke.org/ http://www.pyinvoke.org/
- speg 8y agoPardon my naivety, but what do you normally use?
- giancarlostoro 8y agoI usually do programming in Python and other languages. This particular case it makes sense since I'm taking advantage of command line utilities, otherwise I'm usually just making software myself, e.g. web services or daemons.
- arthulia 8y agoI recommend Greg's Wiki. It's the only resource I use. It covers common pitfalls and anti-patterns. https://mywiki.wooledge.org/ https://mywiki.wooledge.org/
- theonemind 8y agoadvanced bash scripting guide: https://www.tldp.org/LDP/abs/html/ https://www.tldp.org/LDP/abs/html/ also, https://www.shellcheck.net/ https://www.shellcheck.net/ not reading, but pretty neat. static code analysis for shell scripts, points out common errors.
- LukeShu 8y agoI'm a fan of reading code. I would generally describe a GNU/Linux distro as being a "giant pile of shell scripts". That's a little less true with init scripts generally now being systemd units. But that's where I'd start: Look at the code that distros write, that isn't part of some other upstream software. - Arch Linux's `makepkg` https://git.archlinux.org/pacman.git https://git.archlinux.org/pacman.git - Arch Linux's `mkinitcpio` https://git.archlinux.org/mkinitcpio.git/ https://git.archlinux.org/mkinitcpio.git/ - Arch Linux's `netctl` https://git.archlinux.org/netctl.git/ https://git.archlinux.org/netctl.git/ - Downstream from Arch, Parabola's "libretools" dev tools package https://git.parabola.nu/packages/libretools.git/ https://git.parabola.nu/packages/libretools.git/ (disclaimer: I'm the maintainer of libretools) Gentoo also has a lot of good shell scripting to look a, but it's mostly either POSIX shell, or targets older Bash (their guidelines http://devmanual.gentoo.org/tools-reference/bash/index.html http://devmanual.gentoo.org/tools-reference/bash/index.html say to avoid Bash 3 features). I tend to believe that changes made to the Bash language over the years are enhancements, and that they let you write cleaner, more robust code.
- flukus 8y agoBy far my best advice is to write auto-complete scripts if possible: https://iridakos.com/tutorials/2018/03/01/bash-programmable-completion-tutorial.html https://iridakos.com/tutorials/2018/03/01/bash-programmable-... . Many of mine will do things like look up database values, so "command<tab><tab><tab>" is the workflow 99% of the time. Keep them short. There are always exceptions but the "do one thing" mantra is handy, they can always be wrapped into a more complex workflow with a bigger script. None of my frequently used ones are over 100 LoC. Write them for you and you alone when possible, start off simple and iterate. Fight that developer urge to solve a generalized problem for everyone. Embrace the environment and global environment variables. We're trained to avoid global variables like the plague but they are really useful in scripts. My auto complete scripts that I mentioned above, they know which database to connect to based off the environment variable and there are separate commands to switch environment. Make sure you aren't using it where things like make would be more appropriate.
- make3 8y agodoing the hackerrank bash series helped me quite a bit
- jonahx 8y agoGreat video to give you the right high-level perspective: https://www.youtube.com/watch?v=olH-9b3VJfs https://www.youtube.com/watch?v=olH-9b3VJfs
- asicsp 8y agoI have a list of resources related to bash/cli[1] I would highly recommend BashGuide[2] and ryanstutorials[3] as a starting point. After that, go through rest of the wooledge site for FAQs, best practices, etc shellcheck[4] is awesome for checking your scripts for potential downfalls and issues [1] https://github.com/learnbyexample/scripting_course/blob/master/Linux_curated_resources.md https://github.com/learnbyexample/scripting_course/blob/mast... [2] https://mywiki.wooledge.org/BashGuide https://mywiki.wooledge.org/BashGuide [3] https://ryanstutorials.net/linuxtutorial/ https://ryanstutorials.net/linuxtutorial/ [4] https://www.shellcheck.net/ https://www.shellcheck.net/
- rb808 8y agoFrom the google Shell Style Guide: "If you are writing a script that is more than 100 lines long, you should probably be writing it in Python instead. " https://google.github.io/styleguide/shell.xml https://google.github.io/styleguide/shell.xml
- h1d 8y agoLOC alone doen't mean much. I have a backup script in fish of 400+ lines of code because it tries to dump data out of different sources like MySQL, PGSQL, InfluxDB, /etc files and others. Just use what feels comfortable achieving the task.
- notabee 8y agoThat guide is really useful for making scripts more readable.
- shpx 8y agoI recommend reading and memorizing all 30-ish of the readline shortcuts https://tiswww.case.edu/php/chet/readline/readline.html#SEC13 https://tiswww.case.edu/php/chet/readline/readline.html#SEC1... Readline is the library bash uses for editing the input line, and it has some nice movement keys. For example Alt-b moves the cursor back a word, Ctrl-u deletes to the beginning of the line, Ctrl-w removes one word behind the cursor. They work in a bunch of other programs, like the Python interpreter's interactive mode for example.
- Tepix 8y agoExcellent advice! Thanks.
- h1d 8y agoYou could learn bash to understand existing scripts but you could also learn fish, which can be a more sane looking script than bash, if you're writing your own.
- zwischenzug 8y agoYou can’t beat the man page if you’re patient with it. Or you could try my book: https://leanpub.com/learnbashthehardway https://leanpub.com/learnbashthehardway
- unhammer 8y agoRun https://www.shellcheck.net/ https://www.shellcheck.net/ on all your bash code Read https://mywiki.wooledge.org/BashGuide https://mywiki.wooledge.org/BashGuide
- xorcist 8y agoThere used to be this great resource called Linux Documentation Project. It's not as active as it used to be but it produced some really book-quality documents, including the Advanced Bash Scripting Guide at https://www.tldp.org/LDP/abs/html/ https://www.tldp.org/LDP/abs/html/ . Read it! It's great. But know that a lot of bash scripting isn't really in bash, it's really required to be proficient with grep, sed, cut, dc and a few other text processing utilities. Learn those! Then are are a few other tricks to be mindful of: be mindful of spaces (wrap your variable substitution in double quotes, mostly), be mindful of sub-shells (piping to something that sets variables can be problematic), and a few other things that can really only be learned by reading good code. But it's also good to know when you shouldn't venture further. My rule of thumb is when I'm using arrays or hash maps, then it's a good idea to move to another language. That's probably Python nowaways. A lot of people use tons of awk or perl snippets inside their bash scripts, that can also be a sign that it's time to move the whole script over.
- posix_me_less 8y agoAdvanced Bash Scripting Guide on TLDP is a verbose and cumbersome-to-read writeup. Better read the bash man page and then http://mywiki.wooledge.org/BashGuide http://mywiki.wooledge.org/BashGuide and http://wiki.bash-hackers.org/start http://wiki.bash-hackers.org/start
- dvdgsng 8y ago* look into parameter substitution https://www.tldp.org/LDP/abs/html/parameter-substitution.html https://www.tldp.org/LDP/abs/html/parameter-substitution.htm... * use traps http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_12_02.html http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_12_02.htm... * read about safe ways to do things in bash https://github.com/anordal/shellharden/blob/master/how_to_do_things_safely_in_bash.md https://github.com/anordal/shellharden/blob/master/how_to_do... * this is pretty helpful writeup of general CLI usage, yet not bash specific https://github.com/jlevy/the-art-of-command-line https://github.com/jlevy/the-art-of-command-line (related HN discussion https://news.ycombinator.com/item?id=9720813 https://news.ycombinator.com/item?id=9720813)
- wlib 8y agoWhy did we keep the language of the shell and the OS separate? It seems like a needless abstraction which creates more harm than good (read a shell script vs any other language). While I'm at it, why is the filesystem and syscall api not just part of a standard userland language? For example, the filesystem could be exposed like an object tree rather than some syscall ritual. The syscalls could just be invisible, where the language compiler deals with it instead of the programmer. I think that the old LISP machines got this right while we are stuck in a usless POSIX compatibility trap. The only reason I think they didn't design unix this way was because C was too low level, but we could write the OS in a "higher level" functional language.
- techntoke 8y ago> The only reason I think they didn't design unix this way was because C was too low level, but we could write the OS in a "higher level" functional language. You've already lost my interest. Bash/Shell is incredibly powerful and doesn't need a higher-level abstraction. That is what programming languages and CLI tools are for.
- wlib 8y agoThat misses my poorly written point. Do you use a separate, awkward tool to call functions you wrote in python? Or do you call the function in python itself? The shell is a useless abstraction that was only necessary in unix because the alternative was c. Now that we have better languages, why not use them to write the OS bottom-up?
- minitech 8y agoBecause writing new OSes is hard and writing new OSes where existing software that people want to use keeps working is even harder.
- AnIdiotOnTheNet 8y agoAnd thus the tech industry piled abstraction upon abstraction, decade after decade, until finally all software collapsed into a singularity and destroyed the earth. Which, frankly, came as somewhat of a relief to all the people forced to use it.
- BurritoAlPastor 8y agoI can’t imagine what BASH_ARGV0 is for. Can someone more sage supply an example of what problem it solves?
- spoondan 8y agoThe “set” command lets you override the positional arguments but starting at $1. You cannot directly set $0. But you can set BASH_ARGV0, and the value of $0 changes accordingly.
- mturmon 8y ago“BASH_ARGV0: a new variable that expands to $0 and sets $0 on assignment.” Changing argv[0] would make utilities like ps show a more descriptive/shorter name, eg in the case of long command paths.
- dualbus 8y agoI don't think changing argv[0] in the current process will have any effect in the /proc file system. And to do what you describe, there's `exec -a NAME' already: $ (exec -a NOT-BASH bash -c 'echo $0; ps -p $BASHPID -f') NOT-BASH UID PID PPID C STIME TTY TIME CMD dualbus 18210 2549 0 19:30 pts/1 00:00:00 NOT-BASH -c echo $0; ps -p $BASHPID -f
- LukeShu 8y ago> I don't think changing argv[0] in the current process will have any effect in the /proc file system. Yes it does. This is a standard trick for changing the process name at runtime, several daemons do this to change the process name of child processes created by fork() that aren't separate executable. For instance, OpenSSH's sshd sets the child-process for a session to "sshd: USERNAME [priv]". `exec -a` lets you set argv[0] through an execve() call, but many times you want to set it without exec'ing a new program.
- chrisseaton 8y ago> I don't think changing argv[0] in the current process will have any effect in the /proc file system. Yes it does - that’s the whole point of changing it.
- fxfan 8y agoAs someone who lives in zsh and bash for interactive usage- I want to say- please do not write scripts in bash or zsh. Use powershell- its an amazingly well designed scripting language. Also- there is ammonite. Written for scripting.
- flatline 8y agoPowershell has been around for quite a while now, but only recently could you rely on it being installed on Windows systems, let alone be available on Linux. Maybe in another decade, but if you are targeting Unix-like systems bash is probably still your safest bet for portable, interpreted code. Python2 is a close second.
- emersion 8y agoPOSIX sh is your safest bet for portable, interpreted code.
- deleted 8y ago[deleted]
- gnulinux 8y agoWhat percent of Unices come with Powershell today? I can write my code in bash and forget about it. Or if I'm supporting less systems (most linux + macOS + most modern Unices) I can write my code in perl or python. Powershell? Thanks but no thanks.
- int_19h 8y ago> I can write my code in bash and forget about it. In practice, this approach often results in code that works only on Linux, or on Linux and macOS. I especially hate it when people shove #!/bin/bash as a shebang, and doubly so when it's in scripts that are a part of some npm package. (On BSDs, Bash is not installed out of the box, and when it is installed, it's not in /bin, since it's not in the base system.) Still, though, that's a good reason to stick to the original Bourne shell. Powershell, whatever its advantages may be, simply has too heavy dependencies on Linux, since you're dragging in CLR. At that point, like you said, might as well just use Perl or Python for scripting.
- pornel 8y agoReminder that 2019 version of macOS ships with 2007 (last GPL2) version of Bash, and will never ship with any newer version. /bin/bash --version GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin18) Copyright (C) 2007 Free Software Foundation, Inc. macOS used to be an awesome developer machine with good tools out of the box. Now the built-in tools are just a bootstrap for their own replacement via Homebrew. Like IE for downloading Chrome.
- stock_toaster 8y agoIt's just GPLv3 software though. In contract, SSH is currently reporting OpenSSH_7.9p1, and that is fairly up to date. I do think that if Apple can't ship a current version (for whatever reason), they probably shouldn't have it pre-installed at all, much like the BSD's don't come with bash by default either. Maybe they could ship with ksh/pdksh/mksh or something instead.
- hestefisk 8y agoMacOS used to ship with tcsh as its default shell for years.
- pstuart 8y agoI still don't understand how a gnu binary could taint their OS. They can provide the source on opensource.apple.com. Case closed.
- Carpetsmoker 8y agoThe problem is that GPL3 has more restrictions than just "publish the code". That's why projects such as FreeBSD, OpenBSD, etc. also won't include GPL3 code, as including GPL3 code would restrict what you can and can't do with the entire system. I assume that Apple's reasoning is similar.
- ajross 8y agoCan you name them? It's not exactly snark. My experience is that almost everyone with strong GPLv3 opinions turns out not to actually object to the terms themselves when discussed in isolation. To head off: it's not the patent grant. Apache 2 has a very similar patent grant and everyone is fine with it.
- nerdponx 8y agoWhat is the value in creating built-in replacements for binaries like rm and stat?
- chrisshroba 8y agoIt seems like this would eliminate the need to read a file from disk and fork a new process, both of which take time. If you're just removing a single file, this is probably negligible, but if you have a script iterating over 10k files, i.e., this speed-up may be more welcome.
- viraptor 8y ago> if you have a script iterating over 10k files That's probably a good sign you should advance from shell. If the script is trivial, it's going to be trivial in python / ruby / go / crystal / ... as well. If it's not trivial, that's another reason to move.
- pwg 8y agoTrue, but if you are iterating over 10k files and removing them, then a find|xargs pipe with xargs feeding rm the maximum number of parameters the kernel allows per fork will likely be faster than a bash interpreter loop, even with a bash builtin rm.
- e12e 8y ago
- _kst_ 8y agoWith this release, bash now has three built-in variables (um, I mean "parameters") whose values are updated every time they're read: $RANDOM yields a random integer in the range 0..32767. (This feature was already there.) $EPOCHSECONDS yields the whole number of seconds since the epoch. $EPOCHREALTIME yields the number of seconds since the epoch with microsecond precision. I'm thinking of a new shell feature that would allow the user to define similar variables. For example, I have $today set to the current date in YYYY-MM-DD format, and I have to jump through some minor hoops to keep it up to date. Does anyone else think this would be useful enough to propose as a new bash feature? Would it create any potential security holes? Should variables like $PATH be exempted? (Of course this doesn't add any new functionality, since I could use "$(date +%F)" in place of "$today". It's just a bit of syntactic sugar.)
- deleted 8y ago[deleted]
- deleted 8y ago[deleted]
- justizin 8y ago$RANDOM is new in bash 5.0? i'm curious. This has been documented for ages, afaik. headtilt
- sirjaz 8y agoI know I may be downvoted/flamed on this, but why doesn't everyone start looking at powershell as the default shell. All the default parameters you are looking for are already there. Plus, you can you use all the other standard shell tools
- offmycloud 8y agoIt's sad that lists.gnu.org is running obsolete TLS 1.0 crypto with weak 1024-bit DH. Either upgrade to TLS 1.2 with reasonable cipher suites, or just go back to plain HTTP.
- ltc5505 8y agoSince I'm clueless on the subject, can I ask how you determined that information and what resource I could use to become better informed?
- thedanbob 8y agoYou can find that information on the security tab of your browser's developer tools. You can also find a lot more info about any site's HTTPS configuration at ssllabs.com. Here's the report for lists.gnu.org: https://www.ssllabs.com/ssltest/analyze.html?d=lists.gnu.org https://www.ssllabs.com/ssltest/analyze.html?d=lists.gnu.org
- dueyfinster 8y agothis video might help you [1]. [1]: https://www.youtube.com/watch?v=xA_IBcoQTD4 https://www.youtube.com/watch?v=xA_IBcoQTD4
- offmycloud 8y agoA good first step is disabling SSL 3.x and TLS 1.0 in your daily browser. I would also recommend the excellent Qualys SSL Server Test: https://www.ssllabs.com/ssltest/ https://www.ssllabs.com/ssltest/
- avarun 8y agoIs there any way to do that on Chrome macOS?
- nurettin 8y agoBASH_ARGV0 < does that mean we can set process title after the script starts?
- majewsky 8y agoIt would appear so: > New features [...] BASH_ARGV0: a new variable that expands to $0 and sets $0 on assignment.
- timvisee 8y agoYet macOS is still on 3.2.
- cntlzw 8y agoGNU bash, version 3.2.57(1)-release (x86_64-apple-darwin18)
- Tsiklon 8y agoMacOS doesn't appear to ship GPLv3 licensed code. 3.2 is the last update on GPLv2. Alternatively - newer versions of ZSH are frequently provided by Apple.
- adtac 8y agoWhat's wrong with shipping GPLv3 code? Can't they just provide the source (are they making significant changes that they want to keep proprietary?) to comply with the license?
- jordigh 8y agoApple likes to push DRM, which GPLv3 forbids. Apple also is afraid to give patent grants, which GPLv3 requires. Thus, Apple refuses to get anywhere near GPLv3 code, and won't even make it easy to give you GPLv3 source that you can build yourself.
- crehn 8y agobrew install bash
- pokerwe1 8y agoFreechip just For You http://wishmindr.com/list/4dvh#.XDB3syS-FEo http://wishmindr.com/list/4dvh#.XDB3syS-FEo
- wicket 8y agoSeeing this release makes me cringe. I've used Bash as an interactive shell for decades but really I'm sick and tired of it. As a scripting language, I loathe it and really don't understand its purpose. I always write shell scripts in POSIX shell for portability reasons. Most of the time I don't need to use any of Bash's features. In cases where advanced features are needed and portability is not a concern, there are other scripting languages much better suited for this (Python, Ruby, etc). As an interactive shell, the only features I ever use are command history and tab completion. Bash is way too bloated for my use case (it's only a matter of time before the next Shellshock is discovered). Other lightweight shells are missing the couple of interactive features which I do use. If anyone knows of a shell which meets my criteria of being lightweight but with command history and tab completion (paths, command names and command arguments), I'd really appreciate any suggestions. Otherwise I may have to look into extending dash or something.
- manquer 8y agozsh has much better tab completion, you should check it out
- sabujp 8y agohow about a good way to pass around associative arrays and arrays
- ris 8y ago> The `history` builtin ... understands negative arguments as offsets from the end of the history list At last!