6 ms·

Catch: A modern, C++-native, header-only, framework for unit tests

95 points by tbrock 12y ago

rdw 12y ago
This looks hella good, I particularly am intrigued by the nested setup/teardown system. The natural assert syntax is interesting, too, amazing what macros can accomplish. Appropriately, the first question that came to mind was, "what's the catch?" :)
- tbrock 12y ago
  Yup. I found it interesting after a co-worker pointed it out to me because it looks and works kind of like RSpec. We are currently considering using it for a new project and it seems mature enough to give it a go. I'm currently toying with writing a JSONReporter so we can use that as an additional test result output format and I agree, so far it seems to be a really well done project.
  - wichert 12y ago
    FWIW I've been using Catch for a few years now, and I find it mature enough for real world usage.
    - tbrock 12y ago
      Good to know, thanks!
  - rdw 12y ago
    The biggest turnoff to me is the sheer quantity of code (regardless of the fact that it can all be crammed into a single header file). It seems much more difficult to fix a bug in it than in something like UnitTest++. That said, inline setup/teardown and parsing expressions in asserts seem like such natural ideas, I can't believe I haven't run across frameworks doing them before. I guess the shadow of JUnit is extremely long.
- andralex 12y ago
  The macro trick is at https://github.com/philsquared/Catch/blob/785db43bb2cd64bfe795ba1c6adef8b82b85430d/include/internal/catch_capture.hpp#L31 https://github.com/philsquared/Catch/blob/785db43bb2cd64bfe7...: planting the tested expression to the right of an operator ->*, which is overloadable and allows for changing the type of the left hand side in the comparison. The rest is a "simple matter of programming". Clever but probably a bit cute - errors would look weird if the expression isn't set up as expected.
  - ximeng 12y ago
    Also uses template to overload operator ->* https://github.com/philsquared/Catch/blob/785db43bb2cd64bfe795ba1c6adef8b82b85430d/include/internal/catch_result_builder.h https://github.com/philsquared/Catch/blob/785db43bb2cd64bfe7...
larssonvomdach 12y ago
A coworker of mine is working on something similar: https://github.com/Kosta-Github/cute https://github.com/Kosta-Github/cute
X-Istence 12y ago
This looks very intriguing. Right now I am using Google Test for my projects, but a single header file for creating tests is awesome :-)
voltagex_ 12y ago
I'm not a C++ programmer but this looks really neat and tidy. How does it work?
- astrodust 12y ago
  Any sufficiently advanced C++ template code is indistinguishable magic.
chadaustin 12y ago
I don't really see the advantage of Catch over http://unittest-cpp.sourceforge.net/ http://unittest-cpp.sourceforge.net/ UnitTest++ was written by Noel Llopis, is small, tasteful, fast, and supports fixtures (and nested fixtures) via structs and struct inheritance. Honest question: Why would I use Catch instead?
- stinos 12y ago
  I'm in the same boat (having used UnitTest++ for years) and I wonder the same, though I do see some points already: - adding a custom main() to unittest++ was pretty messy - unittest++ is not single-header, meaning you have to recompile all flavours everytime you switch toolsets - that nested sections stuff looks good and readable, so even though it might be doable with unittest++, this really attracts me more That being said, I would like to hear from the author or anyone else who can compare them. But likely I'm going to use this for the next project just to check it out.
- imron 12y ago
  I'm a big fan of UnitTest++, but one thing that would tempt me about Catch is the single header file.
- coderjames 12y ago
  I use Catch over UnitTest++ because I found it first and it was dead simple to get started with. We don't do unit-testing as part of our official development process where I work, so I was looking for something that I could quickly try out and get started with in my spare moments. I read about several frameworks and Catch seemed to fit the bill best. Looking over the UnitTest++ web site just now it appears to be nearly as easy as Catch, so if I had found it first I probably would have given it a try. For those not currently doing any unit testing who want to get a feel for what it is all about, I highly recommend Catch as a "my first unit testing framework."
rikkus 12y ago
A similar thing I knocked up a few years ago. Lacking in features and, well, most things - except it does have a pretty Qt based runner: http://www.rikkus.info/cplusplus-unit-testing-framework http://www.rikkus.info/cplusplus-unit-testing-framework
c_ehlen 12y ago
This looks really good. Do you plan to include BeforeEach and AfterEach? I'll try this out and maybe move my projects to this. I started working on something similar[1] a few days ago just as a proof of concept. [1] https://github.com/cEhlen/CUT/blob/master/example.cpp https://github.com/cEhlen/CUT/blob/master/example.cpp
Svenstaro 12y ago
To be honest, I prefer bandit[1], it uses lambdas instead of macros which I consider more modern and it has a nice syntax to it. Anyone interested in catch should also check out bandit. [1] http://banditcpp.org/index.html http://banditcpp.org/index.html
- wtracy 12y ago
  I found the Bandit examples rather disorienting. Can anyone explain what the "[&](){" business means?
  - vowelless 12y ago
    That's the syntax for describing a lambda function. CPP reference has a nice overview of it: http://en.cppreference.com/w/cpp/language/lambda http://en.cppreference.com/w/cpp/language/lambda
- tveita 12y ago
  One advantage of Catch is that you can use native C++ expressions in your assertions, so instead of AssertThat(guitar->sound(), Equals(sounds::clean)); you can write CHECK(guitar->sound() == sounds::clean); And you'll still get sensible output when it fails, showing the expression and the result of guitar->sound().
  - banachtarski 12y ago
    This isn't so much an advantage as a preference. See https://github.com/joakimkarlsson/snowhouse#assertions https://github.com/joakimkarlsson/snowhouse#assertions
borland 12y ago
I really like the look of the "Sections" for managing setup/teardown of tests. I'm envious and next time I'm hacking on our in-house C# test framework I'm going to try very hard to implement them! Neat!
yoanizer 12y ago
Great work. I love the fact that this encourages writing tests that are readable/documenting. Definitely gonna give this a try. Any recommended test runner to go with it?
- awestroke 12y ago
  There's a test runner included as a macro that defines main
talles 12y ago
Sorry for the off-topicness but I have to say... I loved the project logo
pvidler 12y ago
It looks like a nice test framework, but it doesn't appear to help at all with the really hard problems you face with testing C++ -- isolating the unit under test for separate compilation, and the development and inclusion of mocks/stubs. Anyone know of a really good (preferably simple and pragmatic) C++ mocking/stubbing framework to go with this?
- Ono-Sendai 12y ago
  Why do you want to compile the unit test separately?
  - pvidler 12y ago
    Because that's what makes it a unit test -- you only want the unit, the tests and any stubs/mocks in the executable. Anything more and you have an integration test on your hands. Pulling a single unit out of a legacy codebase can be difficult, especially when you don't want to actually move, copy, or modify the source-files; stubbing out the dependencies and writing mock objects to verify your expectations of what the code should be doing are also very time-consuming. Those are the hard parts of unit testing C++, and I wonder if anyone has found a good way of doing it (with a framework, or just a technique).
npsimons 12y ago
Not to steal your thunder, but TUT (Templated Unit Tests) does this as well, and I've found it works very nicely even in embedded and cross-platform: https://mrzechonek.github.io/tut-framework/ https://mrzechonek.github.io/tut-framework/ It's lightweight, easy to setup for separated tests or built in tests, and simple to use. Doesn't add anything to help with mocking, but you can do that yourself. EDIT: To followup, I created a C++ template that leverages TUT and binfmtc that I could use to rapidly iterate with. I used it for working out solutions to problems in "Thinking in C++"; it's posted at http://hardcorehackers.com/~npsimons/Template.hh http://hardcorehackers.com/~npsimons/Template.hh

95 points by tbrock 12y ago

rdw 12y ago

This looks hella good, I particularly am intrigued by the nested setup/teardown system. The natural assert syntax is interesting, too, amazing what macros can accomplish. Appropriately, the first question that came to mind was, "what's the catch?" :)

tbrock 12y ago
Yup. I found it interesting after a co-worker pointed it out to me because it looks and works kind of like RSpec. We are currently considering using it for a new project and it seems mature enough to give it a go. I'm currently toying with writing a JSONReporter so we can use that as an additional test result output format and I agree, so far it seems to be a really well done project.
- wichert 12y ago
  FWIW I've been using Catch for a few years now, and I find it mature enough for real world usage.
  - tbrock 12y ago
    Good to know, thanks!
- rdw 12y ago
  The biggest turnoff to me is the sheer quantity of code (regardless of the fact that it can all be crammed into a single header file). It seems much more difficult to fix a bug in it than in something like UnitTest++. That said, inline setup/teardown and parsing expressions in asserts seem like such natural ideas, I can't believe I haven't run across frameworks doing them before. I guess the shadow of JUnit is extremely long.
andralex 12y ago
The macro trick is at https://github.com/philsquared/Catch/blob/785db43bb2cd64bfe795ba1c6adef8b82b85430d/include/internal/catch_capture.hpp#L31 https://github.com/philsquared/Catch/blob/785db43bb2cd64bfe7...: planting the tested expression to the right of an operator ->*, which is overloadable and allows for changing the type of the left hand side in the comparison. The rest is a "simple matter of programming". Clever but probably a bit cute - errors would look weird if the expression isn't set up as expected.
- ximeng 12y ago
  Also uses template to overload operator ->* https://github.com/philsquared/Catch/blob/785db43bb2cd64bfe795ba1c6adef8b82b85430d/include/internal/catch_result_builder.h https://github.com/philsquared/Catch/blob/785db43bb2cd64bfe7...

larssonvomdach 12y ago

A coworker of mine is working on something similar: https://github.com/Kosta-Github/cute https://github.com/Kosta-Github/cute

X-Istence 12y ago

This looks very intriguing. Right now I am using Google Test for my projects, but a single header file for creating tests is awesome :-)

voltagex_ 12y ago

I'm not a C++ programmer but this looks really neat and tidy. How does it work?

astrodust 12y ago
Any sufficiently advanced C++ template code is indistinguishable magic.

chadaustin 12y ago

I don't really see the advantage of Catch over http://unittest-cpp.sourceforge.net/ http://unittest-cpp.sourceforge.net/ UnitTest++ was written by Noel Llopis, is small, tasteful, fast, and supports fixtures (and nested fixtures) via structs and struct inheritance. Honest question: Why would I use Catch instead?

stinos 12y ago
I'm in the same boat (having used UnitTest++ for years) and I wonder the same, though I do see some points already: - adding a custom main() to unittest++ was pretty messy - unittest++ is not single-header, meaning you have to recompile all flavours everytime you switch toolsets - that nested sections stuff looks good and readable, so even though it might be doable with unittest++, this really attracts me more That being said, I would like to hear from the author or anyone else who can compare them. But likely I'm going to use this for the next project just to check it out.
imron 12y ago
I'm a big fan of UnitTest++, but one thing that would tempt me about Catch is the single header file.
coderjames 12y ago
I use Catch over UnitTest++ because I found it first and it was dead simple to get started with. We don't do unit-testing as part of our official development process where I work, so I was looking for something that I could quickly try out and get started with in my spare moments. I read about several frameworks and Catch seemed to fit the bill best. Looking over the UnitTest++ web site just now it appears to be nearly as easy as Catch, so if I had found it first I probably would have given it a try. For those not currently doing any unit testing who want to get a feel for what it is all about, I highly recommend Catch as a "my first unit testing framework."

rikkus 12y ago

A similar thing I knocked up a few years ago. Lacking in features and, well, most things - except it does have a pretty Qt based runner: http://www.rikkus.info/cplusplus-unit-testing-framework http://www.rikkus.info/cplusplus-unit-testing-framework

c_ehlen 12y ago

This looks really good. Do you plan to include BeforeEach and AfterEach? I'll try this out and maybe move my projects to this. I started working on something similar[1] a few days ago just as a proof of concept. [1] https://github.com/cEhlen/CUT/blob/master/example.cpp https://github.com/cEhlen/CUT/blob/master/example.cpp

Svenstaro 12y ago

To be honest, I prefer bandit[1], it uses lambdas instead of macros which I consider more modern and it has a nice syntax to it. Anyone interested in catch should also check out bandit. [1] http://banditcpp.org/index.html http://banditcpp.org/index.html

wtracy 12y ago
I found the Bandit examples rather disorienting. Can anyone explain what the "[&](){" business means?
- vowelless 12y ago
  That's the syntax for describing a lambda function. CPP reference has a nice overview of it: http://en.cppreference.com/w/cpp/language/lambda http://en.cppreference.com/w/cpp/language/lambda
tveita 12y ago
One advantage of Catch is that you can use native C++ expressions in your assertions, so instead of AssertThat(guitar->sound(), Equals(sounds::clean)); you can write CHECK(guitar->sound() == sounds::clean); And you'll still get sensible output when it fails, showing the expression and the result of guitar->sound().
- banachtarski 12y ago
  This isn't so much an advantage as a preference. See https://github.com/joakimkarlsson/snowhouse#assertions https://github.com/joakimkarlsson/snowhouse#assertions

borland 12y ago

I really like the look of the "Sections" for managing setup/teardown of tests. I'm envious and next time I'm hacking on our in-house C# test framework I'm going to try very hard to implement them! Neat!

yoanizer 12y ago

Great work. I love the fact that this encourages writing tests that are readable/documenting. Definitely gonna give this a try. Any recommended test runner to go with it?

awestroke 12y ago
There's a test runner included as a macro that defines main

talles 12y ago

Sorry for the off-topicness but I have to say... I loved the project logo

pvidler 12y ago

It looks like a nice test framework, but it doesn't appear to help at all with the really hard problems you face with testing C++ -- isolating the unit under test for separate compilation, and the development and inclusion of mocks/stubs. Anyone know of a really good (preferably simple and pragmatic) C++ mocking/stubbing framework to go with this?

Ono-Sendai 12y ago
Why do you want to compile the unit test separately?
- pvidler 12y ago
  Because that's what makes it a unit test -- you only want the unit, the tests and any stubs/mocks in the executable. Anything more and you have an integration test on your hands. Pulling a single unit out of a legacy codebase can be difficult, especially when you don't want to actually move, copy, or modify the source-files; stubbing out the dependencies and writing mock objects to verify your expectations of what the code should be doing are also very time-consuming. Those are the hard parts of unit testing C++, and I wonder if anyone has found a good way of doing it (with a framework, or just a technique).

npsimons 12y ago

Not to steal your thunder, but TUT (Templated Unit Tests) does this as well, and I've found it works very nicely even in embedded and cross-platform: https://mrzechonek.github.io/tut-framework/ https://mrzechonek.github.io/tut-framework/ It's lightweight, easy to setup for separated tests or built in tests, and simple to use. Doesn't add anything to help with mocking, but you can do that yourself. EDIT: To followup, I created a C++ template that leverages TUT and binfmtc that I could use to rapidly iterate with. I used it for working out solutions to problems in "Thinking in C++"; it's posted at http://hardcorehackers.com/~npsimons/Template.hh http://hardcorehackers.com/~npsimons/Template.hh