4 ms·
it is when the focus is on having 100% coverage and not on test quality. When 100% becomes the metric it tends to get gamed pretty heavily with tests that have
by ender341341 4y ago
it is when the focus is on having 100% coverage and not on test quality. When 100% becomes the metric it tends to get gamed pretty heavily with tests that have such a huge amount of mocks as to make the tests useless, or doing something to make it hit a branch, but ignoring actually verifying it cause that branch is just a log statement.
- latchkey 4y agoI seriously doubt it is actually 100%... let's verify that number.
- ender341341 4y agoon our team it was more "Don't lower it, get accolades for increasing it", and someone gamed it up to 100% using the methods I described.
- yazaddaruvala 4y agoWhen I was at Amazon, the general consensus was 90-95% but not higher. We also excluded all of our Java Spring/Guice configs.
- latchkey 4y agoJust what I thought, thanks for the confirmation. HN loves to take a number and just run with it, without verifying anything.
- dimmke 4y agoNo, my requirement was actually 100%. I have no reason to lie about this. It’s set at a package level.
- latchkey 4y agoWas this just you or everyone on your team?
- dimmke 4y agoeverybody on my team. Are you seriously implying I was put on some kind of special "extra testing required" probation system where the unit test coverage requirement was upped from 95% to 100%? Are you trolling?
- latchkey 4y agoIt sounds kind of absurd really. I've never once been asked to sign a contract that required me to do 100% code coverage. As others have noted in the thread, it is like some mythical number any way. It sounds to me like this wasn't going to end well since the expectations were wonky from the start.
- dimmke 4y agohttps://jestjs.io/docs/configuration#coveragethreshold-object https://jestjs.io/docs/configuration#coveragethreshold-objec... It is absolutely possible to set the coverage threshold in Jest to 100% for all categories and they are set to that in the codebase I worked on. I would spend hours trying to that last 0.02% covered sometimes.
- latchkey 4y agoThe mythical portion of the number is that last 0.02%... the stuff that is just extra work for no logical benefit. It does make me wonder why it would take you hours to do that though... was the code that complicated to test for some reason?
- dimmke 4y agoYes, it was. It used a very obscure language feature of JavaScript. I won't get more detailed than that.
- deleted 4y ago[deleted]
- int_19h 4y agoIt's quite possible to get line coverage to 100%. It doesn't really mean anything because it might be covering those lines getting executed in one very particular order - and all bets are off if that order is different.
- mnd999 4y agoThat was the biggest red flag for me. It’s basically impossible to achieve without writing tests for the sake of it.
- slipshady 4y agoI work closely with the CR-time coverage/enforcement tool. The tool expects 70% new line coverage. The number was chosen arbitrarily, but individual teams/managers can choose to avoid the rule or set a different threshold. It's not ridiculous to expect some, *configurable*, amount of test coverage for newly generated code, is it?
- ender341341 4y agoNo, reasonable levels of testing are part of the job, it's when you get hard and fast rules from leadership like "don't lower coverage, reward increasing coverage" that it starts breaking down, we had that rule and someone gamed it using the methods I described up to 100% coverage.