3 ms·

"Amount of unit tests is pretty small comparing to regression tests. One reason behind that decision is that LLVM internals constantly change all the time. " W

"Amount of unit tests is pretty small comparing to regression tests. One reason behind that decision is that LLVM internals constantly change all the time. " Well, no, and i'm not sure where they got this. It's not a "decision". BTW, This also assumes that all of the tests in the unit test dir are unit tests, and all the tests in test/ are regression tests. This is explicitly false. The general viewpoint of LLVM is that most things people write (analysis, optimization) should be tested by checking the IR or dumps. That is, we want easy to follow and modify unit tests, not just compilable C++ code ones. Thus, you will find a lot of unit tests that are in test/, instead of unittest/, because they are testable using IR printers. Those things that are API's should be tested by unittests, and work is underway to increase coverage. The reason some things aren't cover is simply that in the earlier days of the project, not all of this got unit tests, and it's only in the past couple years that people have become sticklers about it. So that one would be knowingly "not enforcing good development practices", not "llvm internals change all the time". The fact that internals change all the time is in fact a good reason to have unit tests.

Joky 11y ago
The lit test are not really "unitests" per-se, even if we try to consider them as it. We are writing them to stress a specific part of the compiler (can we vectorize this specific construct for instance), but they still involve a large chunk of the compiler. The reason we don't have more real unittests is IMO because: - it is hard to write (you need to mockup the analysis result that is used by the vectorizer if you want to test it in isolation). - it adds lot of maintenance burden/overhead - it does not add much value for the compiler over the lit tests.
- DannyBee 11y ago
  "The reason we don't have more real unittests is IMO because: - it is hard to write (you need to mockup the analysis result that is used by the vectorizer if you want to test it in isolation)" Right, and it's infinitely easier to just dump the result of the analysis and check that it said what you want using lit. We even have a framework explicitly to do that -analyze for opt is specifically meant to run analysis and dump their output. This doesn't test API boundaries, but the dumpers and printers are usually forced to be written in terms of the API. (and folks are starting to get religion about testing the other API boundaries)

Joky 11y ago

The lit test are not really "unitests" per-se, even if we try to consider them as it. We are writing them to stress a specific part of the compiler (can we vectorize this specific construct for instance), but they still involve a large chunk of the compiler. The reason we don't have more real unittests is IMO because: - it is hard to write (you need to mockup the analysis result that is used by the vectorizer if you want to test it in isolation). - it adds lot of maintenance burden/overhead - it does not add much value for the compiler over the lit tests.

DannyBee 11y ago
"The reason we don't have more real unittests is IMO because: - it is hard to write (you need to mockup the analysis result that is used by the vectorizer if you want to test it in isolation)" Right, and it's infinitely easier to just dump the result of the analysis and check that it said what you want using lit. We even have a framework explicitly to do that -analyze for opt is specifically meant to run analysis and dump their output. This doesn't test API boundaries, but the dumpers and printers are usually forced to be written in terms of the API. (and folks are starting to get religion about testing the other API boundaries)