4 ms·
> passes the tests but they have not actually tested Surely this is a problem with the tests then? Whatever manual steps you're referring to with "actually tes
by kaashif 3y ago
> passes the tests but they have not actually tested
Surely this is a problem with the tests then? Whatever manual steps you're referring to with "actually tested" should be automated and part of the test suite.
Much easier said than done in some cases obviously, but I've seen some cases where people were manually doing things to "actually test" their code that could obviously and easily be automated.
- makeitdouble 3y agoI can relate to the issue: tests work with specific check conditions, and there might other side effects that are not critical to the test but still impacting the user. For instance a page that properly loads, but surprisingly slowly. If you had no requirement of speed and it still loads within the test timeouts, it will pass fine, but a manual check would have raised the issue and perhaps underlying well hidden bugs.
- masklinn 3y ago> Surely this is a problem with the tests then? Whatever manual steps you're referring to with "actually tested" should be automated and part of the test suite. As far as I know there is no real way to automate ensuring that the code being submitted is tested. Coverage gates are the closest, and coverage is routinely decried on this here site. The only thing that actually comes close is a very strong type system obviating most tests… but not all, and we’re back to square one. Then there’s the issue of tests testing something useful, yet not over-testing.