7 ms·

Do you work for Anandtech? If not, what are you basing these claims on? I suspect that Anandtech, etc would not publicly disclose if a hardware manufacturer was

Do you work for Anandtech? If not, what are you basing these claims on? I suspect that Anandtech, etc would not publicly disclose if a hardware manufacturer was forbidding them from benchmarking certain configurations under threat of not releasing samples..

wtallis 7y ago
Yes, I write for AnandTech (paid as an independent contractor; I'm not one of the salaried editors). I've done some Spectre/Meltdown regression testing for AnandTech, and I've never been instructed to not do such testing in the future. Microcode benchmarking is not the hill Intel wants to die on.
- BubRoss 7y ago
  Which begs the question, when will we see benchmarks of Intel's CPUs with all vulnerability mitigations on vs. AMDs CPUs?
  - wtallis 7y ago
    As soon as the definition of "with all vulnerability mitigations on" stays stable long enough to put together a good review. Benchmarking a moving target is hell, and we don't have enough equipment or staff to do the around-the-clock regression testing that would be necessary to keep our benchmark database current with everything that's happened over the past 1.5 years.
    - deleted 7y ago
      [deleted]
    - Filligree 7y ago
      People are using your benchmarks to decide what computer to buy. If they're that out of date, what should I tell them?
      - oarsinsync 7y ago
        To use the best information available, which are estimates on what the potential impact can be, and then make their own decisions.
      - wtallis 7y ago
        End-user perceived performance is usually not affected enough to meaningfully change the ranking of products. If a chip goes from being 5% faster to 3% slower when mitigations are applied, you'll never notice that without busting out a stopwatch and digging for a reason to be disappointed. Remember, measurable performance differences aren't always noticeable performance differences, especially without a side by side comparison. And if two competing processors are close enough in performance for these mitigations to change which one comes out on top of benchmark charts, then other factors like price, power consumption and IO capabilities are probably a much bigger deal at that point than minor CPU performance differences. Most if not all of our benchmark suites have been updated to include at least the early Spectre/Meltdown/et al. mitigations, and new CPUs are being tested with the microcode they launch with.
        BuckRogers 7y ago
        Why not do an article with then-current mitigations every 6 months? Rather than conveniently waiting until Intel can get their hardware fixes out. Which will coincide with the "mitigations are stable" article. I'd be willing to bet that if the majority of mitigations impacted AMD rather than Intel as it does today, you'd already have done this and would continue to follow up. This is very interesting material and a special moment in time to cover it and inform your readers. Other than willful laziness ("lack of time", everyone knows you make time for priorities), this appears like shilling hard for Intel. If Anandtech decides to do the right thing, I'd like to see .Net or Java compilation. Real-world based benchmarking only.
        wtallis 7y ago
        > I'd be willing to bet that if the majority of mitigations impacted AMD rather than Intel as it does today, you'd already have done this and would continue to follow up. Fuck you, too. I've given you reasonable explanations and you're still throwing out insulting conspiracy theories. If you want sensationalized news, there are plenty of outlets that will give you what you want, and you don't need to be a dick to those of us who are trying to be reasonable and honest about both the subject matter and the resources we have to provide quality coverage. Also, we've done two significant articles in the past year measuring the impact of these mitigations, so we're not even falling behind the standards you claim to want us to meet.

wtallis 7y ago

Yes, I write for AnandTech (paid as an independent contractor; I'm not one of the salaried editors). I've done some Spectre/Meltdown regression testing for AnandTech, and I've never been instructed to not do such testing in the future. Microcode benchmarking is not the hill Intel wants to die on.

BubRoss 7y ago
Which begs the question, when will we see benchmarks of Intel's CPUs with all vulnerability mitigations on vs. AMDs CPUs?
- wtallis 7y ago
  As soon as the definition of "with all vulnerability mitigations on" stays stable long enough to put together a good review. Benchmarking a moving target is hell, and we don't have enough equipment or staff to do the around-the-clock regression testing that would be necessary to keep our benchmark database current with everything that's happened over the past 1.5 years.
  - deleted 7y ago
    [deleted]
  - Filligree 7y ago
    People are using your benchmarks to decide what computer to buy. If they're that out of date, what should I tell them?
    - oarsinsync 7y ago
      To use the best information available, which are estimates on what the potential impact can be, and then make their own decisions.
    - wtallis 7y ago
      End-user perceived performance is usually not affected enough to meaningfully change the ranking of products. If a chip goes from being 5% faster to 3% slower when mitigations are applied, you'll never notice that without busting out a stopwatch and digging for a reason to be disappointed. Remember, measurable performance differences aren't always noticeable performance differences, especially without a side by side comparison. And if two competing processors are close enough in performance for these mitigations to change which one comes out on top of benchmark charts, then other factors like price, power consumption and IO capabilities are probably a much bigger deal at that point than minor CPU performance differences. Most if not all of our benchmark suites have been updated to include at least the early Spectre/Meltdown/et al. mitigations, and new CPUs are being tested with the microcode they launch with.
      - BuckRogers 7y ago
        Why not do an article with then-current mitigations every 6 months? Rather than conveniently waiting until Intel can get their hardware fixes out. Which will coincide with the "mitigations are stable" article. I'd be willing to bet that if the majority of mitigations impacted AMD rather than Intel as it does today, you'd already have done this and would continue to follow up. This is very interesting material and a special moment in time to cover it and inform your readers. Other than willful laziness ("lack of time", everyone knows you make time for priorities), this appears like shilling hard for Intel. If Anandtech decides to do the right thing, I'd like to see .Net or Java compilation. Real-world based benchmarking only.
        wtallis 7y ago
        > I'd be willing to bet that if the majority of mitigations impacted AMD rather than Intel as it does today, you'd already have done this and would continue to follow up. Fuck you, too. I've given you reasonable explanations and you're still throwing out insulting conspiracy theories. If you want sensationalized news, there are plenty of outlets that will give you what you want, and you don't need to be a dick to those of us who are trying to be reasonable and honest about both the subject matter and the resources we have to provide quality coverage. Also, we've done two significant articles in the past year measuring the impact of these mitigations, so we're not even falling behind the standards you claim to want us to meet.