10 ms·

Bash_unit – bash unit testing framework

102 points by arunc 6y ago

chovybizzass 6y ago
haha. i love it!
atsaloli 6y ago
See also https://github.com/sstephenson/bats https://github.com/sstephenson/bats
- ice3 6y ago
  I'd recommend checking out this fork instead: https://github.com/bats-core/bats-core https://github.com/bats-core/bats-core
  - atsaloli 6y ago
    Nice! Glad to know development is continuing. Thanks!
- williamdclt 6y ago
  I used bats a while ago to test a homemade CLI to open github pull-requests, it worked well enough for me (https://github.com/williamdclt/git-pretty-pull-request https://github.com/williamdclt/git-pretty-pull-request if you're looking for an example, although I did not maintain the tests out of laziness so they are very red). Cannot compare it to bash_unit, but I'm happy there's alternatives!
  - dnautics 6y ago
    Yeah i think most people looking for this don't necessarily care about the distinction between unit/integration (i myself don't really know anymore because so many of my elixir "exunit" tests hit the db -- it's fine) they just want something TAP compliant and maybe also offers green dots.
lelanthran 6y ago
Funny, you can read the title as an instruction and it still sorta makes sense :-)
- ahmetkun 6y ago
  I guess another good example of that would be lobash. Assuming you're familiar with lodash, of course.
louwrentius 6y ago
I appreciate the effort but I think that if you are unit testing shell scripts, I think it’s probably better to use a language like Python or something similar. I am serious. I have decided for my own efforts to use python for any kind of command line script and I’m always glad I did.
- fnord123 6y ago
  bats is used for running tests against command line interfaces of your program. This is useful for making sure weird strings and argument patterns are handled correctly.
- loloquwowndueo 6y ago
  I still find it a bit awkward to use python just to run some commands in sequence with some massaging of input/output data and parameters based on simple logic. bash scripts are great for that. And in that case it’s still a good idea to have automated testing instead of relying on running the script a few times to ensure it behaves as it should. I’ll definitely give this a try :) I do agree though that if you need extensive massaging of output or arguments, python can help and make the whole thing easier.
  - disgruntledphd2 6y ago
    The trouble is that bash is necessary to setup python. For instance, the lingua franca of Docker is bash (well sh, but you should obviously change that). I've been resisting learning (any more) bash for a while now, but I think I'm going to have to. This looks like it could reduce some of my terror at the insanities of shell scripts (why does one even need two quote characters that do different things?)
    - sethammons 6y ago
      One set of quotes (double quotes) allows for interpolation of commands/variables. Try: $ echo ‘pwd’ $ echo “pwd”
CraigJPerry 6y ago
There’s clearly some good thought gone into this - i especially like the ergonomics of the “fake” command and arguments. However, having been down this path myself, i don’t think it’s a great idea. Beyond the hello world examples you start to run into the need to defang commands being tested or getting better visibility of what they’re doing just so you can make a useful assert, you end up LD_PRELOAD’ing shims to intercept calls and it gets a bit horrifying, very quickly.
- deleted 6y ago
  [deleted]
- deleted 6y ago
  [deleted]
- acdha 6y ago
  I agree: this is great as an intellectual exercise but I’m reminded about how many times I’ve been glad that I set a policy in the 2000s that any bash script too large for a normal size screen should be rewritten in Python.
  - fooflarney 6y ago
    or just make
    - CraigJPerry 6y ago
      GNU make? BSD make? (Trick question - which BSD make?) nmake? (Another trick question...) - and that’s just Linux, MacOS and Windows today. Admittedly it was even worse previously. There’s so many flavours of make with annoyingly incompatible syntax and that’s before you get into the presence of GMSL or equiv on each platform. There’s only one fitting answer to this and it’s the output of the following command: make love Except depending on your make, this joke doesn’t work :-(
JackuB 6y ago
If you fancy more BDD-style testing with a Gherkin-like syntax, there is also Shellspec https://shellspec.info https://shellspec.info / https://github.com/shellspec/shellspec https://github.com/shellspec/shellspec
- ryanianian 6y ago
  Shellspec is delightful to use. Every time I go to test with it I end up finding bugs. But also every time I go to test with it I end up convincing myself to not use bash. I imagine the same would be true of bash_unit.
justinholmes 6y ago
Shunit2 as well https://github.com/kward/shunit2 https://github.com/kward/shunit2
actionowl 6y ago
If you fancy posix shell (and C/C++ support) https://github.com/jmmv/atf/ https://github.com/jmmv/atf/
oweiler 6y ago
I've built a Bash testing framework myself, and my realization was that Bash is a bad language for this. https://github.com/helpermethod/bash-specs https://github.com/helpermethod/bash-specs
a1o 6y ago
Hey, any ideas how to leverage your current testing framework to add coverage or if there's any pretty way to print coverage results that are somehow as standard as TAP?
- bewuethr 6y ago
  I have successfully used https://github.com/infertux/bashcov https://github.com/infertux/bashcov in the past.
jasonpeacock 6y ago
For everyone who's saying "if you need unit tests then you should rewrite it in a different language", you're not wrong but you're not right either. Bash is a lowest-common-denominator language, available on almost all platforms, with a very stable (old?) API, which makes it ideal for broad distribution & bootstrapping of systems. When you use another other languages, like Python, suddenly you're not only worrying about whether Python is installed, but also which version is available, and you still can't use anything that's not part of the standard library (because then you enter package management/dependency hell). I agree that even Python with stdlib-only is still better than Bash, but I guarantee that you'll find Bash >=3.0 on every host.
- titzer 6y ago
  I used bash to write a testing framework for a project I have that needs minimal dependencies. E.g. I don't want to depend on having Python (and the transitive closure of its batteries-included experience) installed. For that purpose, bash works pretty well. I also want to run on some older machines, like 32-bit macs. Turns out that bash does evolve...slowly. E.g. I found out that the default bash on MacOS 10.4 doesn't support the "=~" operator...oi...
  - arendtio 6y ago
    In addition, the default MacOS Bash is buggy :-/
- hiq 6y ago
  > I agree that even Python with stdlib-only is still better than Bash, but I guarantee that you'll find Bash >=3.0 on every host. Shouldn't you go for POSIX at that point? I'm not an expert, but I still have some idea how to test that my script is (somewhat) POSIX compliant (e.g. use shellcheck, run the script with dash or a shell in POSIX-compatible mode). I have Bash 5 on my Debian, how do I test that a script I tested with this version works on Bash 3 as well? Or have the changes between Bash 3 and Bash 5 been so minimal that I would have to do it on purpose to find an incompatibility? I'm also curious about the number of machines where Bash scripts are run but don't have, say, Python 3.4 installed, and projects where it's a better trade-off to spend time developing and testing in Bash rather than have a Python 3 dependency. I'd say it's relatively rare.
prussian 6y ago
When I was writing my IRC Bot In Bash, I was looking for something akin to this. for the most part I got away with simple mocking for the IRC bits, but all the functionality (plugins & commands) is still untested. Around the time I created it bats was a thing, but it didn't seem that unobtrusive. this seems more in line with how I'd probably do testing for bash, though at this point I'm not so sure what I'd use it for. Maybe It's worth going over some personal or work projects and trying this out.
jascii 6y ago
On one hand, like a lot of posters are saying, maybe use another language if it gets complicated enough to require unit testing. On the other hand, a lot of the stuff that gets written in bash is quite mission critical, so increasing test coverage is a good thing!
- pdkl95 6y ago
  On the 3rd hand, a major aspect of the 'engineering' part of being a 'software engineer' is learning how to choose which side of various tradeoffs is appropriate for a project. Should we perform the expensive computation on demand at runtime, or is trading time for space with a lookup table more appropriate? Is the task simple enough for a basic 'glue language' like bash, or does it require a full-featured language like ruby or python? Regardless, as you pointed out, regardless of the language some sort of testing is still a good idea. Even small, supposedly 'trivial' scripts deserve some testing!
synergy20 6y ago
posix sh, be it dash or ash, can do 90% of whatever bash does and is way more portable.
- chimpansteve 6y ago
  I'm not sure I understand this. Those shells only have 90% of bash functionality, but are somehow more portable? I get the POSIX argument, but I'm not sure I agree. In my experience at least, POSIX is less supported than bash. I'm pretty confident that any random terminal I have to ssh into will have a bash shell. I ssh into a lot of terminals. I know where you're coming from, it just doesn't chime with my personal experience.
  - synergy20 6y ago
    on embedded boards which are deployed 100x more than desktops and are typically resource restricted you have to use posix shell instead of the mega-bytes bash there.

102 points by arunc 6y ago

chovybizzass 6y ago

haha. i love it!

atsaloli 6y ago

See also https://github.com/sstephenson/bats https://github.com/sstephenson/bats

ice3 6y ago
I'd recommend checking out this fork instead: https://github.com/bats-core/bats-core https://github.com/bats-core/bats-core
- atsaloli 6y ago
  Nice! Glad to know development is continuing. Thanks!
williamdclt 6y ago
I used bats a while ago to test a homemade CLI to open github pull-requests, it worked well enough for me (https://github.com/williamdclt/git-pretty-pull-request https://github.com/williamdclt/git-pretty-pull-request if you're looking for an example, although I did not maintain the tests out of laziness so they are very red). Cannot compare it to bash_unit, but I'm happy there's alternatives!
- dnautics 6y ago
  Yeah i think most people looking for this don't necessarily care about the distinction between unit/integration (i myself don't really know anymore because so many of my elixir "exunit" tests hit the db -- it's fine) they just want something TAP compliant and maybe also offers green dots.

lelanthran 6y ago

Funny, you can read the title as an instruction and it still sorta makes sense :-)

ahmetkun 6y ago
I guess another good example of that would be lobash. Assuming you're familiar with lodash, of course.

louwrentius 6y ago

I appreciate the effort but I think that if you are unit testing shell scripts, I think it’s probably better to use a language like Python or something similar. I am serious. I have decided for my own efforts to use python for any kind of command line script and I’m always glad I did.

fnord123 6y ago
bats is used for running tests against command line interfaces of your program. This is useful for making sure weird strings and argument patterns are handled correctly.
loloquwowndueo 6y ago
I still find it a bit awkward to use python just to run some commands in sequence with some massaging of input/output data and parameters based on simple logic. bash scripts are great for that. And in that case it’s still a good idea to have automated testing instead of relying on running the script a few times to ensure it behaves as it should. I’ll definitely give this a try :) I do agree though that if you need extensive massaging of output or arguments, python can help and make the whole thing easier.
- disgruntledphd2 6y ago
  The trouble is that bash is necessary to setup python. For instance, the lingua franca of Docker is bash (well sh, but you should obviously change that). I've been resisting learning (any more) bash for a while now, but I think I'm going to have to. This looks like it could reduce some of my terror at the insanities of shell scripts (why does one even need two quote characters that do different things?)
  - sethammons 6y ago
    One set of quotes (double quotes) allows for interpolation of commands/variables. Try: $ echo ‘pwd’ $ echo “pwd”

CraigJPerry 6y ago

There’s clearly some good thought gone into this - i especially like the ergonomics of the “fake” command and arguments. However, having been down this path myself, i don’t think it’s a great idea. Beyond the hello world examples you start to run into the need to defang commands being tested or getting better visibility of what they’re doing just so you can make a useful assert, you end up LD_PRELOAD’ing shims to intercept calls and it gets a bit horrifying, very quickly.

deleted 6y ago
[deleted]
deleted 6y ago
[deleted]
acdha 6y ago
I agree: this is great as an intellectual exercise but I’m reminded about how many times I’ve been glad that I set a policy in the 2000s that any bash script too large for a normal size screen should be rewritten in Python.
- fooflarney 6y ago
  or just make
  - CraigJPerry 6y ago
    GNU make? BSD make? (Trick question - which BSD make?) nmake? (Another trick question...) - and that’s just Linux, MacOS and Windows today. Admittedly it was even worse previously. There’s so many flavours of make with annoyingly incompatible syntax and that’s before you get into the presence of GMSL or equiv on each platform. There’s only one fitting answer to this and it’s the output of the following command: make love Except depending on your make, this joke doesn’t work :-(

JackuB 6y ago

If you fancy more BDD-style testing with a Gherkin-like syntax, there is also Shellspec https://shellspec.info https://shellspec.info / https://github.com/shellspec/shellspec https://github.com/shellspec/shellspec

ryanianian 6y ago
Shellspec is delightful to use. Every time I go to test with it I end up finding bugs. But also every time I go to test with it I end up convincing myself to not use bash. I imagine the same would be true of bash_unit.

justinholmes 6y ago

Shunit2 as well https://github.com/kward/shunit2 https://github.com/kward/shunit2

actionowl 6y ago

If you fancy posix shell (and C/C++ support) https://github.com/jmmv/atf/ https://github.com/jmmv/atf/

oweiler 6y ago

I've built a Bash testing framework myself, and my realization was that Bash is a bad language for this. https://github.com/helpermethod/bash-specs https://github.com/helpermethod/bash-specs

a1o 6y ago

Hey, any ideas how to leverage your current testing framework to add coverage or if there's any pretty way to print coverage results that are somehow as standard as TAP?

bewuethr 6y ago
I have successfully used https://github.com/infertux/bashcov https://github.com/infertux/bashcov in the past.

jasonpeacock 6y ago

For everyone who's saying "if you need unit tests then you should rewrite it in a different language", you're not wrong but you're not right either. Bash is a lowest-common-denominator language, available on almost all platforms, with a very stable (old?) API, which makes it ideal for broad distribution & bootstrapping of systems. When you use another other languages, like Python, suddenly you're not only worrying about whether Python is installed, but also which version is available, and you still can't use anything that's not part of the standard library (because then you enter package management/dependency hell). I agree that even Python with stdlib-only is still better than Bash, but I guarantee that you'll find Bash >=3.0 on every host.

titzer 6y ago
I used bash to write a testing framework for a project I have that needs minimal dependencies. E.g. I don't want to depend on having Python (and the transitive closure of its batteries-included experience) installed. For that purpose, bash works pretty well. I also want to run on some older machines, like 32-bit macs. Turns out that bash does evolve...slowly. E.g. I found out that the default bash on MacOS 10.4 doesn't support the "=~" operator...oi...
- arendtio 6y ago
  In addition, the default MacOS Bash is buggy :-/
hiq 6y ago
> I agree that even Python with stdlib-only is still better than Bash, but I guarantee that you'll find Bash >=3.0 on every host. Shouldn't you go for POSIX at that point? I'm not an expert, but I still have some idea how to test that my script is (somewhat) POSIX compliant (e.g. use shellcheck, run the script with dash or a shell in POSIX-compatible mode). I have Bash 5 on my Debian, how do I test that a script I tested with this version works on Bash 3 as well? Or have the changes between Bash 3 and Bash 5 been so minimal that I would have to do it on purpose to find an incompatibility? I'm also curious about the number of machines where Bash scripts are run but don't have, say, Python 3.4 installed, and projects where it's a better trade-off to spend time developing and testing in Bash rather than have a Python 3 dependency. I'd say it's relatively rare.

prussian 6y ago

When I was writing my IRC Bot In Bash, I was looking for something akin to this. for the most part I got away with simple mocking for the IRC bits, but all the functionality (plugins & commands) is still untested. Around the time I created it bats was a thing, but it didn't seem that unobtrusive. this seems more in line with how I'd probably do testing for bash, though at this point I'm not so sure what I'd use it for. Maybe It's worth going over some personal or work projects and trying this out.

jascii 6y ago

On one hand, like a lot of posters are saying, maybe use another language if it gets complicated enough to require unit testing. On the other hand, a lot of the stuff that gets written in bash is quite mission critical, so increasing test coverage is a good thing!

pdkl95 6y ago
On the 3rd hand, a major aspect of the 'engineering' part of being a 'software engineer' is learning how to choose which side of various tradeoffs is appropriate for a project. Should we perform the expensive computation on demand at runtime, or is trading time for space with a lookup table more appropriate? Is the task simple enough for a basic 'glue language' like bash, or does it require a full-featured language like ruby or python? Regardless, as you pointed out, regardless of the language some sort of testing is still a good idea. Even small, supposedly 'trivial' scripts deserve some testing!

synergy20 6y ago

posix sh, be it dash or ash, can do 90% of whatever bash does and is way more portable.

chimpansteve 6y ago
I'm not sure I understand this. Those shells only have 90% of bash functionality, but are somehow more portable? I get the POSIX argument, but I'm not sure I agree. In my experience at least, POSIX is less supported than bash. I'm pretty confident that any random terminal I have to ssh into will have a bash shell. I ssh into a lot of terminals. I know where you're coming from, it just doesn't chime with my personal experience.
- synergy20 6y ago
  on embedded boards which are deployed 100x more than desktops and are typically resource restricted you have to use posix shell instead of the mega-bytes bash there.