5 ms·

For C and C++, what's wrong with using assert() in an integration_test() function or many <func>_test() functions, called when passed a --test flag or compiled

by vortico 10y ago

For C and C++, what's wrong with using assert() in an integration_test() function or many <func>_test() functions, called when passed a --test flag or compiled as a separate binary? For me, benchmarking isn't really useful unless it's a profiler in a real world use situation. Otherwise you have no idea how often each API function is called.

pg314 10y ago
Nothing. That's the way I usually test my C code. It has the drawback that one failing test aborts the whole program. The error messages are not the most informative, but it does the job.
- klodolph 10y ago
  This is easy enough to solve on Unix... simply fork() before each test. Just make sure that the master is single-threaded or this won't work.
  - gpderetta 10y ago
    Also an option is to siglongjmp out of the abort signal handler.
- vortico 10y ago
  Although hacky, assert(test_foo() /* Checks the sanity of Foo */); does the job for the error messages, and replacing abort() with a custom function like `void check(int condition, const char *format, ...);` can make it not abort the program on an error. I prefer the regular assert() though because I want the build to fail by an aborting test program, and all I care is the line number of failure. Another benefit is that abort() will pause gdb in the exact place that it crashed, with the stack intact if you want to inspect it.
  - cormacrelf 10y ago
    You can also use the comma operator to document your assert()s, if you don't like using comments. assert(("test_foo() should be true", test_foo())); https://en.wikipedia.org/wiki/Comma_operator#Asserts https://en.wikipedia.org/wiki/Comma_operator#Asserts
    - gpderetta 10y ago
      I have used often and also seen in the wild the following form: assert(condition && "message");
- codr4life 10y ago
  I've found it worthwhile to add a thin facade on top of assert to support out-of-band reporting of errors even if NDEBUG is disabled: https://github.com/codr4life/libc4/blob/master/src/c4/assert.h https://github.com/codr4life/libc4/blob/master/src/c4/assert...
koja86 10y ago
For C++ I find both Google Test and Boost Unit Test Framework pretty usable. They both have some quirks but IMHO no show-stoppers. I prefer these to custom solutions because don't want to maintain yet another tool but to focus on my problem domain and it is nice when some fellow developers actually have experience with these tools.
- thedonkeycometh 10y ago
  I think thats the main rub, think about the maintainers. Using a bespoke test framework, is often a way of introducing conflict, no matter how good a coder you are. As Jeff Atwood says, imagine the maintainer being a psychopath.. https://blog.codinghorror.com/coding-for-violent-psychopaths/ https://blog.codinghorror.com/coding-for-violent-psychopaths...
  - codr4life 10y ago
    Would you rather have your psychopath stalkers tearing their hair out over the Boost dependency you dragged in? Once written it stays mostly the same, there is not much to maintain. I can't see why rolling your own framework would introduce conflicts; unless you're in an environment where writing code to solve problems is frowned upon, but then you have worse problems.
- gpderetta 10y ago
  I prefer Boost.UnitTest to Google Test, which I also like. Boost.UnitTest has very nice sugar, but the big problem is the very slow compilation times.
  - imron 10y ago
    That's why I like catch [0] Super easy to use, header only so no need to compile anything beforehand, no external dependencies, reasonably fast compilation time, and does everything I need. 0: https://github.com/philsquared/Catch https://github.com/philsquared/Catch
    - 10y ago
codr4life 10y ago
There's nothing wrong with assert, that's what I use for checking conditions as well. The framework posted is just a thin facade on top of regular functions and asserts to provide fixtures, dynamic grouping and benchmarks. Knowing how long a specific set of tests take to run for N repetitions is useful information. Since I wrote the test, I know exactly how many times the tested API is called. This is about spotting problems and trends early, once I find something interesting I'll drop down to profiling.

by vortico 10y ago

pg314 10y ago

Nothing. That's the way I usually test my C code. It has the drawback that one failing test aborts the whole program. The error messages are not the most informative, but it does the job.

klodolph 10y ago
This is easy enough to solve on Unix... simply fork() before each test. Just make sure that the master is single-threaded or this won't work.
- gpderetta 10y ago
  Also an option is to siglongjmp out of the abort signal handler.
vortico 10y ago
Although hacky, assert(test_foo() /* Checks the sanity of Foo */); does the job for the error messages, and replacing abort() with a custom function like `void check(int condition, const char *format, ...);` can make it not abort the program on an error. I prefer the regular assert() though because I want the build to fail by an aborting test program, and all I care is the line number of failure. Another benefit is that abort() will pause gdb in the exact place that it crashed, with the stack intact if you want to inspect it.
- cormacrelf 10y ago
  You can also use the comma operator to document your assert()s, if you don't like using comments. assert(("test_foo() should be true", test_foo())); https://en.wikipedia.org/wiki/Comma_operator#Asserts https://en.wikipedia.org/wiki/Comma_operator#Asserts
  - gpderetta 10y ago
    I have used often and also seen in the wild the following form: assert(condition && "message");
codr4life 10y ago
I've found it worthwhile to add a thin facade on top of assert to support out-of-band reporting of errors even if NDEBUG is disabled: https://github.com/codr4life/libc4/blob/master/src/c4/assert.h https://github.com/codr4life/libc4/blob/master/src/c4/assert...

koja86 10y ago

For C++ I find both Google Test and Boost Unit Test Framework pretty usable. They both have some quirks but IMHO no show-stoppers. I prefer these to custom solutions because don't want to maintain yet another tool but to focus on my problem domain and it is nice when some fellow developers actually have experience with these tools.

thedonkeycometh 10y ago
I think thats the main rub, think about the maintainers. Using a bespoke test framework, is often a way of introducing conflict, no matter how good a coder you are. As Jeff Atwood says, imagine the maintainer being a psychopath.. https://blog.codinghorror.com/coding-for-violent-psychopaths/ https://blog.codinghorror.com/coding-for-violent-psychopaths...
- codr4life 10y ago
  Would you rather have your psychopath stalkers tearing their hair out over the Boost dependency you dragged in? Once written it stays mostly the same, there is not much to maintain. I can't see why rolling your own framework would introduce conflicts; unless you're in an environment where writing code to solve problems is frowned upon, but then you have worse problems.
gpderetta 10y ago
I prefer Boost.UnitTest to Google Test, which I also like. Boost.UnitTest has very nice sugar, but the big problem is the very slow compilation times.
- imron 10y ago
  That's why I like catch [0] Super easy to use, header only so no need to compile anything beforehand, no external dependencies, reasonably fast compilation time, and does everything I need. 0: https://github.com/philsquared/Catch https://github.com/philsquared/Catch
  - 10y ago

codr4life 10y ago

There's nothing wrong with assert, that's what I use for checking conditions as well. The framework posted is just a thin facade on top of regular functions and asserts to provide fixtures, dynamic grouping and benchmarks. Knowing how long a specific set of tests take to run for N repetitions is useful information. Since I wrote the test, I know exactly how many times the tested API is called. This is about spotting problems and trends early, once I find something interesting I'll drop down to profiling.