4 ms·
It depends on how junior the engineer is. My first job out of college, I was asked to write some code to cheat a benchmark, basically detect when a particular b
by ryandrake 4y ago
It depends on how junior the engineer is. My first job out of college, I was asked to write some code to cheat a benchmark, basically detect when a particular benchmark program was running and only then put the software into an alternate "fast path" that would result in better benchmark results. I agonized over this and didn't want to refuse. This was my first real job as a professional developer, and I didn't want to make waves. Eventually I got the nerve to tell my boss I was uncomfortable with the assignment, and he said "Oh, no problem at all! We keep our devs happy here." and assigned me onto another task. Joe, three cubicles down was more than happy to write the benchmark-cheating code.
- rtev 4y agoYou did the right thing. If more people spoke up and stood up, the world would be a much happier place.
- sebastiansm 4y agoSome people don’t need to stand up because they don’t see any bad or evil in his work.
- koyote 4y agoJust out of curiosity, why did the software not always use the 'fast path'?
- Infinity315 4y agoProbably more than likely it's not generalizable to real world conditions. Benchmarks are meant to be reproducible, meaning perfectly predictable. CPUs have things called branch predictors which try to predict what the software is going to do and try to do the calculation ahead of time resulting in (hopefully, if it predicted right) faster execution time. If you know which 'branches' a benchmark goes down, you can make a program which can coax the branch predictor to always make the right guesses for a given benchmark. A program branches whenever you encounter some sort of conditional if-else statement.
- rot13xor 4y agoSome optimizations might only apply to certain inputs that are used by benchmarking software. Or the driver makes unfavorable power tradeoffs to maximize performance when a benchmark is running. For example, if the driver knows a benchmark is single threaded, it can artificially throttle other cores and boost the core the benchmark is running on. There's more extreme stuff like GPU drivers replacing shaders (benchmarks don't care about graphics quality) or pre-rendering frames. https://videocardz.com/74912/professional-overclocker-demonstrates-how-to-cheat-in-3dmark-timespy https://videocardz.com/74912/professional-overclocker-demons...
- pixl97 4y agoIf you have the answer card to a test it takes much less 'computation' to get the test correct. Print '2' is lot less operations than (insert formula here) if you know the answer is already 2.