3 ms·

That's a very peculiar way to do unit testing, comparing the output of the whole test instead of specific asserts. How would you compare this with something li

That's a very peculiar way to do unit testing, comparing the output of the whole test instead of specific asserts. How would you compare this with something like Shunit2[1]? I can see someone missing some error confirmations by being careless when producing the output or testing things in isolation. But I can also see how this could be more likely to catch some side effects (stupid example: unclosed terminal escapes) in combined output. I was looking for a nice testing tool for my bash migration tool[2], and while the `assert` way is well known, I like writing bash because it's bare and requires discipline and yours seems survivalist enough. [1]: https://code.google.com/p/shunit2/ https://code.google.com/p/shunit2/ [2]: https://github.com/zenoamaro/rash https://github.com/zenoamaro/rash

txutxu 12y ago
I've seen it on many places to test shell scripts. For example: in the same bash interpreter source code. See files using the '.right' extenion [0]. I think it's a flexible approach, compared to using things as shunit, or TAP producers. The main advantage of using shunit (or TAP producers too), I think is to re-use available tools (at least in the case of TAP producers), and the possible familiarity of new contributors with the syntax. Personally I avoid test libraries that use traps and eval, and bash code can become so complex, that I thing there is not a single testing library (Written in bash) that could work with all my code base. As soon as you start using advanced features (shell options, traps, code tracing features, readonly stuff, filedescriptors, etc) most assumptions become wrong. Anyway, the most hard part of test complex shell programs (specially those related to system administration), is to test functionality. It's easy (and even not useful) to assert that 'myecho 1' will print '1' and return 0. But things become hard to test when you want to see if a partition routine works well with +2TB disks, if a netfilter ruleset is resilient to attacks (unknown by the author), if a program is safe to unexpected (by the programmer) input, if a backup restore will not break production... Fun to see this on hackernews front page. One of my pet projects is just the same, but not being inspired by chef. [0] http://git.savannah.gnu.org/cgit/bash.git/tree/tests http://git.savannah.gnu.org/cgit/bash.git/tree/tests
- alganet 12y ago
  It's funny you mentioned the exact problems I'm trying to solve. For the tests, I'm writing my own xUnit tool [1]. It doesn't use any fancy stuff, runs on set -euf, it's tested by itself and works on every POSIX-like shell that I could find (bash, dash, zsh, (m|pd)ksh, yash, posh, even busybox) and many different versions and OSs. On the functional side, I've decided that I need to test using virtual machines. The test matrix tool that can spawn and test on them is almost ready [2], it already tests the library itself and exports the matrix configuration to Travis. Very early stuff though and doesn't even touch problems like partition routines you mentioned, but overall I'm happy with the results, considering everything runs under 70Kb of code. [1] https://github.com/Mosai/workshop/blob/master/doc/posit.md https://github.com/Mosai/workshop/blob/master/doc/posit.md [2] https://github.com/Mosai/workshop/blob/master/test/matrix.sh https://github.com/Mosai/workshop/blob/master/test/matrix.sh

txutxu 12y ago

I've seen it on many places to test shell scripts. For example: in the same bash interpreter source code. See files using the '.right' extenion [0]. I think it's a flexible approach, compared to using things as shunit, or TAP producers. The main advantage of using shunit (or TAP producers too), I think is to re-use available tools (at least in the case of TAP producers), and the possible familiarity of new contributors with the syntax. Personally I avoid test libraries that use traps and eval, and bash code can become so complex, that I thing there is not a single testing library (Written in bash) that could work with all my code base. As soon as you start using advanced features (shell options, traps, code tracing features, readonly stuff, filedescriptors, etc) most assumptions become wrong. Anyway, the most hard part of test complex shell programs (specially those related to system administration), is to test functionality. It's easy (and even not useful) to assert that 'myecho 1' will print '1' and return 0. But things become hard to test when you want to see if a partition routine works well with +2TB disks, if a netfilter ruleset is resilient to attacks (unknown by the author), if a program is safe to unexpected (by the programmer) input, if a backup restore will not break production... Fun to see this on hackernews front page. One of my pet projects is just the same, but not being inspired by chef. [0] http://git.savannah.gnu.org/cgit/bash.git/tree/tests http://git.savannah.gnu.org/cgit/bash.git/tree/tests

alganet 12y ago
It's funny you mentioned the exact problems I'm trying to solve. For the tests, I'm writing my own xUnit tool [1]. It doesn't use any fancy stuff, runs on set -euf, it's tested by itself and works on every POSIX-like shell that I could find (bash, dash, zsh, (m|pd)ksh, yash, posh, even busybox) and many different versions and OSs. On the functional side, I've decided that I need to test using virtual machines. The test matrix tool that can spawn and test on them is almost ready [2], it already tests the library itself and exports the matrix configuration to Travis. Very early stuff though and doesn't even touch problems like partition routines you mentioned, but overall I'm happy with the results, considering everything runs under 70Kb of code. [1] https://github.com/Mosai/workshop/blob/master/doc/posit.md https://github.com/Mosai/workshop/blob/master/doc/posit.md [2] https://github.com/Mosai/workshop/blob/master/test/matrix.sh https://github.com/Mosai/workshop/blob/master/test/matrix.sh