3 ms·
You want repeatable conditions for tests so that you can compare results between models, and against the norms. When you have repeatable conditions - software
by ajuc 11y ago
You want repeatable conditions for tests so that you can compare results between models, and against the norms.
When you have repeatable conditions - software in the tested product can detec that and act differently in these conditions.
That's exactly what happened in VW case.
It's nontrivial to fix the test so that it is still repeatable and hard to fool by company determined to fool it.
- madez 11y agoRandomized test with a suitable number of runs will cost-efficiently give useable results. I agree, it's not trivial. But, it's not hard either.