3 ms·

I don't see how the act of taking and publishing measurements after microcode and OS updates constitutes hysteria. It's my understanding, at least on Windows, y

I don't see how the act of taking and publishing measurements after microcode and OS updates constitutes hysteria. It's my understanding, at least on Windows, you pretty much have to opt-out of these patches or manually install an update that disables the mitigations. I fully support a users right to bypass these mitigations, and you're correct that your typical desktop user, at least today, isn't a target. But it seems odd that websites dedicated to performance computing have a blindspot to how automatically installed updates will impact performance.

wtallis 7y ago
> I don't see how the act of taking and publishing measurements after microcode and OS updates constitutes hysteria. It's quite easy to sensationalize benchmark results even unintentionally. The average reader of PC hardware review sites is totally willing to latch on to a microbenchmark result that shows a 20% performance drop and claim that it's disastrous for performance, even if the actual added delay to real-world operations is a fraction of a millisecond and thus will almost never cause the result of your user input to be delayed by even a single frame. There's a certain degree of irresponsibility in publishing results that you know will be taken out of context by almost everyone who reads them. I've discontinued benchmarks in the past because it was frustrating seeing readers pretend like they show a meaningful difference between products when the reader's workload never comes close to the workload represented by that benchmark.
- chmod775 7y ago
  Did you just make an argument against PC review sites publishing any benchmarks at all because readers can't be trusted to interpret them correctly? That would kind of defeat the point. If they published a benchmark in the past and don't bother to correct the benchmark when it becomes out of sync with reality - that is just bad journalism. Nobody is saying you should go and cherry-pick benchmarks after the mitigations hit, but you should definitely check the benchmarks you already published once. These sites can and should expect an informed reader. In any case: Leaving wrong information up uncontested helps neither "experts" nor laymen.
  - wtallis 7y ago
    > Did you just make an argument against PC review sites publishing any benchmarks at all because readers can't be trusted to interpret them correctly? No, and you should know better. > If they published a benchmark in the past and don't bother to correct the benchmark when it becomes out of sync with reality - that is just bad journalism. Proper practice is to publish the full test conditions, including software, firmware and nowadays also microcode versions. The availability of newer versions does not make older results any less true. At AnandTech, we make all reasonable attempts to keep a thorough database of older hardware tested on newer benchmark suites, but the time this requires means we cannot re-test everything multiple times per year. I have over 200 SSDs and counting in the collection, and that test suite is over 30 hours long. The collection of CPUs is much larger. GPU reviews typically have fewer back-catalog hardware entries because updating to new drivers a few times a year is often unavoidable. You can browse the results for current and previous test suites at https://www.anandtech.com/bench/ https://www.anandtech.com/bench/ > These sites can and should expect an informed reader. You don't read the comments as often as we do.
- sundvor 7y ago
  In the case of my 6850k my overclock was silently killed by the Windows 10 microcode update which locked multi to 38x. This behaviour angered me no end. I wasted significant time looking for workarounds, and deleting the microprocessor driver was the only way. I wonder what fixes I've now nixed, but there was seriously no need for Intel to kill my overclock. On a couple of occasions Intel have pushed updates which have reset my fix. Dear $deity .. my next PC will be AMD for sure.

wtallis 7y ago

> I don't see how the act of taking and publishing measurements after microcode and OS updates constitutes hysteria. It's quite easy to sensationalize benchmark results even unintentionally. The average reader of PC hardware review sites is totally willing to latch on to a microbenchmark result that shows a 20% performance drop and claim that it's disastrous for performance, even if the actual added delay to real-world operations is a fraction of a millisecond and thus will almost never cause the result of your user input to be delayed by even a single frame. There's a certain degree of irresponsibility in publishing results that you know will be taken out of context by almost everyone who reads them. I've discontinued benchmarks in the past because it was frustrating seeing readers pretend like they show a meaningful difference between products when the reader's workload never comes close to the workload represented by that benchmark.

chmod775 7y ago
Did you just make an argument against PC review sites publishing any benchmarks at all because readers can't be trusted to interpret them correctly? That would kind of defeat the point. If they published a benchmark in the past and don't bother to correct the benchmark when it becomes out of sync with reality - that is just bad journalism. Nobody is saying you should go and cherry-pick benchmarks after the mitigations hit, but you should definitely check the benchmarks you already published once. These sites can and should expect an informed reader. In any case: Leaving wrong information up uncontested helps neither "experts" nor laymen.
- wtallis 7y ago
  > Did you just make an argument against PC review sites publishing any benchmarks at all because readers can't be trusted to interpret them correctly? No, and you should know better. > If they published a benchmark in the past and don't bother to correct the benchmark when it becomes out of sync with reality - that is just bad journalism. Proper practice is to publish the full test conditions, including software, firmware and nowadays also microcode versions. The availability of newer versions does not make older results any less true. At AnandTech, we make all reasonable attempts to keep a thorough database of older hardware tested on newer benchmark suites, but the time this requires means we cannot re-test everything multiple times per year. I have over 200 SSDs and counting in the collection, and that test suite is over 30 hours long. The collection of CPUs is much larger. GPU reviews typically have fewer back-catalog hardware entries because updating to new drivers a few times a year is often unavoidable. You can browse the results for current and previous test suites at https://www.anandtech.com/bench/ https://www.anandtech.com/bench/ > These sites can and should expect an informed reader. You don't read the comments as often as we do.
sundvor 7y ago
In the case of my 6850k my overclock was silently killed by the Windows 10 microcode update which locked multi to 38x. This behaviour angered me no end. I wasted significant time looking for workarounds, and deleting the microprocessor driver was the only way. I wonder what fixes I've now nixed, but there was seriously no need for Intel to kill my overclock. On a couple of occasions Intel have pushed updates which have reset my fix. Dear $deity .. my next PC will be AMD for sure.