10 ms·
static_assert is all you need (no leaks, no UB)
- kris-jusiak 3y agoWith C++20 almost anything can be used in constexpr context (vector, unique_ptr, virtual, function, etc.) and as long as it's in the scope it can be tested at compile time which guarantees memory safatey, no UB, etc. Additionally, since constexpr can be executed at run-time and code has been tested at compile-time already therefore 'static_assert' is (almost) all you need - https://godbolt.org/z/P4cqboGx6 https://godbolt.org/z/P4cqboGx6.
- saagarjha 3y agoHave zero cost abstractions gone too far?
- AshamedCaptain 3y agoI really don't think it works that way. You can for sure write code that will leak and otherwise be UB but run compile-time just fine.
- kris-jusiak 3y agoconstexpr has to checked for leaks and UB so as long as there is coverage at compile-time (static_assert + constexpr) I would assume there shouldn't be neither leaks nor UB. But the context is limitted where that can be applied and actually compiles. For example, there is no way to do it with global variables but with limited scope that's possible.
- AshamedCaptain 3y agoAt the very minimum, you can simply do is_constant_evaluated to change behavior depending on runtime vs compile-time...
- the_mitsuhiko 3y agoI _think_ you cannot conclude from a constexpr not having UB at compile time that it won’t have UB at runtime.
- kris-jusiak 3y agoI guess so, can't think of an example now but I'm pretty sure there are subtle corner cases (as always) and it depends on the testing, coverage and potential limitations of checking things at compile-time, though, IMHO, the technique is promissing and can help with a lot of use cases but defo not everything.
- layer8 3y ago(nevermind)
- jvanderbot 3y agoI think it's worth pointing out that your statement is true by definition, perhaps not true by implementation in these cases. It's not like UB produces _random_ behavior, it's just not specified what compilers _should_ do in those cases. Of course, cannot be relied upon.
- deleted 3y ago[deleted]
- jcelerier 3y agoSuccessful compilation of UB is explicitly disallowed by the standard in a constexpr context
- layer8 3y agoThanks, I wasn’t aware. Although that seems to be restricted to core language UB and not include standard-library UB: https://stackoverflow.com/a/72494688/623763 https://stackoverflow.com/a/72494688/623763
- soulbadguy 3y ago>constexpr has to checked for leaks and UB so as long as there is coverage at compile-time Source ?I am not sure constexpr give any garanty regarding UB and/or leaks
- 3836293648 3y agoConstexpr at compile times gives that guarantee, not constexpr in general. Hence static assert to force comptime evaluation
- soulbadguy 3y ago> Constexpr at compile times gives that guarantee Can you point to a source for that ? I am not trying to be pendantic, but genuilly curious
- dgellow 3y agohttps://en.cppreference.com/w/cpp/language/constant_expression https://en.cppreference.com/w/cpp/language/constant_expressi... Point 8: > A core constant expression is any expression whose evaluation would not evaluate any one of the following: > 8. an expression whose evaluation leads to any form of core language undefined behavior (including signed integer overflow, division by zero, pointer arithmetic outside array bounds, etc).
- layer8 3y agoI hope there’s also some text in the standard prohibiting implementations from allowing any other expressions as a constant expression (which they otherwise could as a language extension), and thus requires compilation failure for such expressions?
- kps 3y agoPragmatically, you can't stop extensions; if the fine print for `--std=cool++23` says that this mode is not actually C++23-compliant, nearly nobody will ever notice or care. Pragmatically, if a popular compiler makes `--std=cool++23` the default, and requires `--std=C++23 --iso-eic-jtc1-sc22-wg21 --pompous` to get standard-conforming behaviour, nearly nobody will do that; instead they will complain that other compilers lack the extensions.
- layer8 3y agoExtensions can be standard-compliant, in the sense that they don’t violate any prescription by the standard, and thus a program cannot assume their absence. My question was whether the standard actually takes care to render the acceptance of constexprs-with-UB non-standard-compliant. That is, in addition to “must accept X”, does it also say “must only accept X”?
- tlb 3y agoIndeed, it fails if you invoke UB. For example, an int overflow in a constexpr causes a compile error: <source>:5:17: error: static assertion expression is not an integral constant expression static_assert(1024*1024*1024*3 != 0, "UB"); https://godbolt.org/z/Wc591En7E https://godbolt.org/z/Wc591En7E
- mikepurvis 3y agoFrom an overall performance point of view, I wonder how the timing works out for the compiler to run your unit tests like this vs to produce and invoke a binary. I bet it's mostly a wash, and the ergonomics of conventional gtest macros look way better to my eye.
- kris-jusiak 3y agoHmm, there are obvisouly trade offs (it depends on the compiler how many tests, how are they written, etc.) but for apples to apples comparision the gtest binary would have to be either compiled with sanitizers (that would be probably slower to compile than static_assert tests without sanitizers) or run with valgrind or similar (execution would be much slower, static_asserts tets don't have to be executed, compiles=green).
- ynik 3y agoThe idea to let the compiler run the tests only works if you constexpr everything, which means putting all code in headers. This effectively means giving up on separate compilation. Worse, if you use some mixture (most code in headers but you still have >1 compilation unit), your compile times completely explode as essentially all code is compiled repeatedly for each unit.
- cozzyd 3y agoThere are other ways to do this. I made a proof of concept of using linker sections to allow you to sprinkle tests within the implementation inline once... https://github.com/cozzyd/examc https://github.com/cozzyd/examc (this is obviously not production-ready, just serves as a proof of concept). Basically the idea is that the test code gets written to a different linker section that your test runner can iterate through, when tests are enabled. This is easy on gcc because it generates automatic constants for the beginning and end of different linker sections. There may be away to do this with clang as well, but I never use clang.
- gpderetta 3y agoPragmatically you can write your code in such a way that you can get immediate feedback in your IDE as you write the code if a static assert fails as you are implementing your function. You could of course set up your runtime tests in a similar way, having the ide run them back to back as you are writing code, but it is more complicated, especially if the code is in an intermediate state that it is not fully compilable. So in the end it is not a huge breakthrough, but having compile time tests is still quite a nice feature.
- zerr 3y agoDepends on how you define a class. E.g. you are not using std::list in your example.
- fefe23 3y agoThis is awesome! I wonder what the compile time cost would be to just have this kind of unit testing in all the time.
- kris-jusiak 3y agoThanks! Most likely not yet applicable at Google's scale but smaller project can defo leverage the approach. Personally, I'm writing most of my tests this way and with TDD the red phase is always a compilation fail which is quicker than buiding and running in my experience. But that's for a medium size project. But as always it depends there are trade offs.
- macgyverismo 3y agoI’ve been using this technique as well, but I found that debugging static_asserts is quite hard. I often fall back to calling the failing test at runtime and stepping through. Any suggestions for a different workflow?
- kris-jusiak 3y agoIMHO the best approach is to avoid the problem by applying TDD. Then there is very little need to debug anything. But otherwise, there is https://github.com/mikael-s-persson/templight https://github.com/mikael-s-persson/templight for compile-time debugging which is pretty cool and having something like `expect(auto... args) static_asert(args...); assert(args...);` may help with being able to debug at run-time and get the coverage (though, the code has has to compile aka pass first).
- rphil 3y ago[dead]
- JTyQZSnP3cQGa8B 3y ago(Offtopic but) Kris, I have used your micro unit-test library in the past and it was a pleasure to look at your code. You're the kind of crazy guy (in a good way) that gives me the motivation to learn new stuff. Thanks.
- deleted 3y ago[deleted]
- dig1 3y agoThis makes sense for straightforward tests, but static_assert is not all you need in general, because some things has be executed in runtime, after series of steps or after some timeframe. Good luck reproducing or testing these in compile-time.
- mr_00ff00 3y agoCould you not put a series of steps or something that mocks time into constexpr? A comment mentioned above that in C++20, almost all features are available at compile time now.
- ovao 3y agoWith few limitations, yes. In C++20, you could for example test for constant evaluation[1], using that as a mechanism to fall back to a real clock during runtime. [1]: https://en.cppreference.com/w/cpp/types/is_constant_evaluated https://en.cppreference.com/w/cpp/types/is_constant_evaluate...
- leni536 3y agoPer standard wording if an evaluation contains an undefined operation then it is not a constant expression. It means that undefined behavior should result in compilation failure if it happens in a context where a constant expression is required, like in static_assert. However the standard only requires this for language-level undefined behavior. For undefined behavior happening in the standard library it's unspecified whether the expression is a constant expression or not. So no, constexpr tests don't cover all possible UB. Also even if in theory language-level undefined behavior should be caught in constant expressions, in practice compilers miss a number of undefined behaviors. They are generally good at catching out-of-range indexing, using objects outside of their lifetime, using uninitialized values, signed integer overflow and modifying const objects. However there are a number of subtle undefined behaviors that they don't catch, like unsequenced operations on the same object, invalid values for unscoped enums. There might be some overlap with runtime tests with -fsanitize=undefined,address. For catching uninitialized values at runtime though you probably need msan, which is a pain to set up, but constexpr tests cover that. On the other hand the function you test might not be available at compile-time. Anyway, constexpr tests are a valuable tool. It's not a silver bullet.
- 0xbkt 3y agoWhat's going on in this code for someone completely alien to C++?
- ksherlock 3y agoThe compiler is doing a bunch of complicated stuff at compile time. In fact it's both a C++ compiler and a C++ interpreter. static_assert is a compile-time check. [] { ... }(); is an immediately executed lambda function (IIFE in javascript parlance). list<int> list{} is a linked list of integers (double-linked, forward and backward). push_back() allocates more memory. pop() / clean() deallocates memory.
- cubancigar11 3y agoIt is creating a bunch of objects, modifying them, then asserting their value all at compile time. For example the first example creates a list and asserts its size is 0. List normally allocated on heap, so I am guessing they have made changes in thay area in c++20 by making it Constexpr, which is a fancy way to say an expression can be known at compile time.
- tialaramex 3y agoThe C++ code is a simple list type, and also a bunch of tests for that type to confirm that it works as intended. The crucial trick here is that because static asserts are used, the test values are computed during compilation, such computations are forbidden (by the standard) from having any leaks or Undefined Behaviour. Anything allocated must be freed by the time the tests complete, and no language Undefined Behaviour is permitted. The latter is pretty normal for other languages but is a big deal in C++ where UB is a constant plague. However many languages have either forbid or have strict limits on compile time heap allocation - after all that heap isn't going to still exist at runtime. Requiring that you free everything allocated fixes that hole and means you get free leak detection.
- rphil 3y ago[dead]
- goldbattle 3y agoInteresting work. Thanks for open sourcing. It would be nice if the "run" script has a bit of documentation, and maybe an "examples" folder the the run script can work with. From what I can see it seems to search the parent directory? I would want for example `./run <dir_with_tests>` to run all tests my specified project repo.
- rphil 3y agoYou can now specify a directory, see README. Any other arguments are just passed to the compiler. Here's a sample to play with: https://github.com/yellowdragonlabs/samples/blob/master/tdd_sample.cpp https://github.com/yellowdragonlabs/samples/blob/master/tdd_...
- mgaunard 3y agoNew user of C++ discovers C++ has a mechanism for code to be evaluated at compile-time, and decides to share his findings on twitter. Why is it a story? Slow day on the Internet?
- gpderetta 3y agoSomeone found ot interesting enough to write about this and someone else found it interesting enough to upvote. In the end it just an excuse to have a discussion on an interesting topic. And yes, it is a slow day.
- mrlonglong 3y agoYou need C++ 23 to run this code. I tried it out with C++ 20 and got compile errors.
- pjmlp 3y agoHence "-std=c++2b" on compiler explorer example.