7 ms·
MinUnit – A minimal unit testing framework for C (2002)
- pavlov 8y agoSo, the revelation is that a unit test is an if statement, and if you wrap that into a macro you can call it a framework.
- ticviking 8y agoFor many novice engineers this is a huge revelation.
- pavlov 8y agoReally? Is it perhaps because of those libraries that do their best to make test call sites look like weird pseudo-English. Stuff like having a library function named it(): it(“should be confusing”, function() { ... })
- collinf 8y agoIt always really bothered me that there had to be so much syntactical sugar on top of testing frameworks. Unit testing should be written in exactly the syntax of the language you are working with. I don't want to have to carry that additional crap in my working memory while writing code. I can't stand having to look up Gherkin syntax just to write a few dumb tests. The only reason I see the usefulness of additional syntax just for tests is the case to have non-technical people writing tests. From my experience, this is a terrible idea.
- davydog187 8y agoOh man, I wonder how your opinions would change if you used a language with great first class testing. Elixir's unit testing framework is part of the language, and is excellent. https://hexdocs.pm/ex_unit/master/ExUnit.html https://hexdocs.pm/ex_unit/master/ExUnit.html Its not really about syntactical sugar, and more about being able to see the data the failed, and make the testing experience easy so that you can write tests with less effort
- a_imho 8y agoOTOH never rule out how KISS is the enemy of billable hours and don't underestimate CDD's ability (consultability driven development) to come up with solutions to non problems. YMMV, the BDD implementations I've seen so far did not convince me of its value to put it mildly.
- bramblerose 8y agoI never understood the value of Gherkin/Cucumber style testing until I worked with a QA engineer. The engineer started writing the tests (in Cucumber format) as a way to write __manual__ tests: they allow you to write 'do this, then that, verify this' in a structured manner. Automating this was a secondary goal: once you have (relatively) structured manual tests, it's nice if you can automate them, while still retaining the possibility to run them manually. This is actually important, and they would typically be end-to-end UI tests, which can be quite fragile when completely automated. Having them in a human-friendly format allows you to have a manual fallback if e.g. a changed identifier blocks you from running the automated test.
- lotyrin 8y agoYep. The value of Given When Then ends, not begins, once you have implemented the steps with some automation.
- sidlls 8y agoYou're describing "fluent programming" ("it().shouldEqual(x)..."). Unfortunately it's not just unit testing frameworks where this nightmare exists. Plenty of production code abuses the builder pattern to have this "it's like reading English" stuff.
- tonysdg 8y agoIt's not really that terrible of a concept -- some of the earliest high-level languages (e.g., COBOL) were designed to be "readable" by less-technical staff. It just requires careful design and structure. IMO, it only falls apart (and becomes such a nightmare) because of the imprecision of spoken & written word. Computers don't understand idioms, colloquialisms, and the like, and so non-technical staff are tricked into thinking a language/program can do more than it actually can do.
- sidlls 8y agoThis sort of programming seems popular with languages that are already verbose (Java), and it just adds pointless extra verbosity.
- phkahler 8y agoAnd any small set of functions is an API.
- hinkley 8y agoExcept that’s not a whole test any more than the punchline is the joke. It’s sample data, verification, analysis, and reporting. The if statement is just the verification step.
- davydog187 8y agoWhats up with the do/while loops? Does that just make the return syntactically valid? Apologies, its been a while since I've touched C/C++
- pdexter 8y agohttps://stackoverflow.com/questions/154136/why-use-apparently-meaningless-do-while-and-if-else-statements-in-macros https://stackoverflow.com/questions/154136/why-use-apparentl...
- tom_ 8y agoThis is dealt with at the bottom of the article. It's a C FAQ: http://c-faq.com/cpp/multistmt.html http://c-faq.com/cpp/multistmt.html
- okl 8y agoGCC has an extension that can be used instead of the do-while loop, see example macro definition here: https://gcc.gnu.org/onlinedocs/gcc/Statement-Exprs.html https://gcc.gnu.org/onlinedocs/gcc/Statement-Exprs.html
- hector_ka 8y agoWorks great in Embedded Linux, and it is easy to crosscompile.
- gon1332 8y agoWell, big minds meet. Here is mine Minimal Unit Testing FW for C: https://github.com/gon1332/minut https://github.com/gon1332/minut
- codeplea 8y agoWhy stick to three lines? With only a couple more you could reduce the boiler-plate needed to use it. I wrote a minimal C test framework too: https://github.com/codeplea/minctest https://github.com/codeplea/minctest It's got some real-world usage. Still only just one header file, but it'll time each test and if an equal assertion fails it'll print the variable values.
- jwilk 8y ago(2002) according to the HTTP headers: $ curl -s --head http://www.jera.com/techinfo/jtns/jtn002.html | grep ^Last Last-Modified: Fri, 06 Sep 2002 07:16:46 GMT
- vortico 8y agoWhy not just 1 line? Of course, the application will halt after one test fails, but some people like it this way. #include <assert.h> Usage: void test_foo() { assert(foo() == 4 /* foo should be 4 */); } Output would be something like Assertion failed at "foo() == 4 /* foo should be 4 */" If you run the application in a debugger like GDB, you can see the frame when the abort trap is called.
- jwilk 8y agoComment won't be included the assertion message. If you want to put arbitrary string there, you need something like this: assert("foo should be 4" && (foo() == 4));
- vortico 8y agoAh, thanks, I had forgotten.
- stevefan1999 8y agoI was about the point this out! At the end of the day, unit testing is just contract assertions, and contract assertions are usually conditions! But actually this isn't really the crux of unit testing, the value of unit testing is to iterate and refactor your program for a more predictable and bug-free application, not just writing a bunch of conditions.
- simias 8y agoI actually implemented a basic unit test framework in C yesterday and started with basic asserts like you point out, however once I got it working I switched to return values not unlike TFA. The obvious problem with assertions in that they kill your test program immediately at the first failure instead of doing additional tests and potentially give a clearer picture of what's broken exactly. They're also not very flexible if you want to customize the error message since they only stringify the assert parameter. "Assertion `ret == 0' failed." is not super helpful. As sibling comments mention there are ways to work around that by using literal strings and && but it doesn't help if you want to wrap around tests using helper functions etc... I'm also tempted to say that if you need to use GDB to figure out where your tests have failed exactly your framework is not particularly user friendly. Normally when a test fails it should be pretty clear where it failed and why. The root cause of the issue might be further away of course but then again assertions won't help with that either.
- tom_ 8y agoOne thing I'm pretty doctrinaire about when it comes to this sort of thing is printing out more than just a simple message. Quite often, this makes the problem obvious with no need for deeper investigation. To do this I have a bunch of macros like this: /* check A and B are equal. */ #define EQ_II(A,B,M) (CheckEQII((A),(B),M,#A,#B,__FILE__,__LINE__)) You use it like this: EQ_II(i,3,"blah blah blah"); CheckEQII looks roughly like this: void CheckEQII(int64_t a,int64_t b,const char *message,const char *a_str,const char *b_str,const char *file,int64_t line) { if(a!=b) { printf("%s:%" PRId64 ": test failed: %s\n",file,line,message); printf(" Values not equal.\n"); printf(" Got expr : %s\n",a_str); printf(" Wanted expr : %s\n",b_str); printf(" Got value : %" PRId64 " (0x%" PRIx64 ")\n"); printf(" Wanted value: %" PRId64 " (0x%" PRIx64 ")\n"); DEBUG_BREAK(); exit(1); } } (DEBUG_BREAK breaks into the debugger if you're running in the debugger.) The FILE:LINE notation is probably clickable in your favourite text editor. (For VC++, use "FILE(LINE):". Just do #ifdef _MSC_VER or something.) Very convenient if you run tests as part of the build. And you can flesh it out for strings, arrays, floats, doubles, and all the rest. You can fit everything you need into about 500 lines. This isn't quite as impressive as the 3 lines here, but compared to something like Catch - which is a huge amount of C++ code, crazy C++ code to boot, that adds literally seconds to your build time - and, no, the fact that seconds is a drop in the ocean in C++land is not an excuse - it's in the same ballpark. At least, its extra utility should prove, over the course of a project, in my view, commensurate with the extra LOC.
- masklinn 8y agoThat's why I love pytest. You just `assert` an expression, and on failure it'll break down each sub-expression and show whatever it yielded, plus by default it captures all output (stdout and stderr) and prints it out only on failure (so logging and other console output is not painfully annoying while testing) and the fixtures systems is just fantastic and parameterised tests make for much clearer data-driven (/table-driven) tests and the tests selection at the CLI is awesome. And all the complexity is in the test framework and runner, all you do is write def test_thing(a_fixture): assert foo(bar) == a_fixture.some_thing() or whatever.
- crescentfresh 8y agoI'm not very familiar with C and macros so it took me a while to realize that this line: mu_assert("error, bar != 5", bar == 5); will return out of the function before reaching the next line. For anyone else like me, I put together the "inlined" version of the code to help me understand what is happening: #include <stdio.h> #include "minunit.h" int tests_run = 0; int foo = 7; static char * test_foo() { do { if (!(foo == 7)) return "error, foo != 7"; } while (0); return 0; } static char * all_tests() { do { char *message = test_foo(); tests_run++; if (message) return message; } while (0); // ... more tests here return 0; }
- giancarlostoro 8y agoAnd then in D it's built-in to the compiler so it's a 1 liner of sorts: unittest { ... } and they all run during compilation. https://dlang.org/spec/unittest.html https://dlang.org/spec/unittest.html
- tonyedgecombe 8y agoI'm surprised that isn't used in other languages, having the test right next to the code seems such a good idea.
- giancarlostoro 8y agoRust went this path too, there's something I love about having unittesting readily available. I can choose to use it or not use it. My asserts are available to me as I so desire. Python too sorta has some unittesting in the standard library, but that's not a compiler level thing.
- blt 8y agowhat happens if you write a really slow test? is there some way to take advantage of the inbuilt console reporting, etc. without running them after every compile?
- giancarlostoro 8y agoIt's not compiled into your binary unless you want it to be. It's ran at compile time every time you compile, though I'm sure there's a flag to not run unit testing, and maybe with version() you might be able to block out the time ocnsuming tests. Not sure what you'd do in D that'd be so time consuming though. Edit: URL to version() docs: https://dlang.org/spec/version.html https://dlang.org/spec/version.html
- Yuioup 8y agoI did something like that once in Classic ASP (VBScript). I think I only wrote two Assert functions and that was enough. The code ended up being really robust after I wrote a bunch of test cases that eventually passed (after writing the code of course).
- VictorSCushman 8y agoI have a library [link redacted] that was originally designed to extend MinUnit. I love the idea of an all-in-one unit testing library without complicated set up.
- mikestew 8y agoWell, there's a fine welcome from HN: new account that's fifteen minutes old, which posts what could be useful information (I just skimmed it) in a non-controversial manner and...insta-dead. I obviously vouched for the comment, but yeesh.
- sctb 8y agoI hear you, but since we don't have software that can determine whether posts follow the guidelines, sometimes (especially for brand new accounts) the anti-abuse software gets it wrong so then we have vouching.
- mikestew 8y agoYeah, I get what you're up against (I do skim New once in a while). Just a good thing someone has Show Dead on and has sufficient points to vouch, eh? :-)
- VictorSCushman 8y agoThank you :)
- okl 8y agoFor everybody who demands from their test framework that it prints expected vs actual values and file/line references, and that it can be adapted to embedded and bare metal targets I recommend the excellent Unity framework[1], example[2]. -- [0] http://www.throwtheswitch.org/unity/ http://www.throwtheswitch.org/unity/ [1] https://gitlab.com/oliver117/blut/blob/master/test/test_math_double_special.c https://gitlab.com/oliver117/blut/blob/master/test/test_math...
- bfrog 8y agoLibcheck does the nice thing and let's you optionally fork each test resulting in parallelism and test failures from segfaults and such not causing the whole suite failing. Does this do something similar?
- blt 8y agoI did mine using assert.h and a signal handler. More code, but advantage is you can also use assert.h internally anywhere in your program without it needing to know about the test framework. On other hand, makes testing optimized builds difficult.
- RhysU 8y agoFCTX is in a similar space, and quite good: https://github.com/imb/fctx/blob/master/README.rst https://github.com/imb/fctx/blob/master/README.rst
- makecheck 8y agoThe examples suggest that maybe the implementation should use “#str” in the macros (i.e. when you want your string to contain the same thing as the expression itself).