4 ms·

I don’t believe these numbers mean what they think. Their sample size has dropped dramatically since the previous year, from 100k to 25k for laptops and from 18

I don’t believe these numbers mean what they think. Their sample size has dropped dramatically since the previous year, from 100k to 25k for laptops and from 186k to 48k for desktops. Given that all the data comes from people choosing to run the benchmark, I wonder what population has suddenly left the data, and if that is significant. Also consider that the CPU is only one component of this benchmark. The article itself says that Windows 11 performance is worse than Windows 10. This might be another instance of “What Andy [Grove] giveth, Bill [Gates] taketh away.”

high_na_euv 2y ago
Still, those sample sizes are huge
- brokensegue 2y ago
  Sample size won't save you if you're sampling from a different population.
  - deleted 2y ago
    [deleted]
  - high_na_euv 2y ago
    Sure, but we dont know
    - crazygringo 2y ago
      If there's an artifact here it's probably seasonality. E.g. a proper comparison would look at what the results were for just January over the past few years. It's easy to imagine that benchmarks in January might sample more from cheaper Christmas present laptops, while the rest of the year has more powerful professional purchases spread throughout. Or after Christmas upgrades, people get hand-me-down computers and disproportionately benchmark those to figure out what they're good for. You can invent a million scenarios, but it's extremely plausible that January results could be an outlier every year.
- kazinator 2y ago
  Sample size cannot compensate for sampling bias.
crazygringo 2y ago
> Their sample size has dropped dramatically since the previous year, from 100k to 25k for laptops and from 186k to 48k for desktops. I suspect that's just an effect of 2025 data being limited to just ~January, rather than a full 12 months. If people run a benchmark only once every 4 months on average, that would certainly explain the sample size.
- sonofhans 2y ago
  Yeah, good point. Or maybe it means people got slightly less-performant systems for Christmas than last year. Gates’ Law finally catching up with us all.
  - deleted 2y ago
    [deleted]
veqq 2y ago
> “What Andy [Grove] giveth, Bill [Gates] taketh away.” Also known as https://en.wikipedia.org/wiki/Wirth%27s_law https://en.wikipedia.org/wiki/Wirth%27s_law
rcarmo 2y ago
I think there are several factors at play here, starting with “my current machine is good enough” and lower PC hardware sales as well as indeed, a bit of a shift to lower-power, quieter, smaller machines, and CPU characteristics changing (with more e-cores). Example: The mini-PC market has exploded, and people buying those are neither benchmark enthusiasts nor an active part of their readership.

high_na_euv 2y ago

Still, those sample sizes are huge

brokensegue 2y ago
Sample size won't save you if you're sampling from a different population.
- deleted 2y ago
  [deleted]
- high_na_euv 2y ago
  Sure, but we dont know
  - crazygringo 2y ago
    If there's an artifact here it's probably seasonality. E.g. a proper comparison would look at what the results were for just January over the past few years. It's easy to imagine that benchmarks in January might sample more from cheaper Christmas present laptops, while the rest of the year has more powerful professional purchases spread throughout. Or after Christmas upgrades, people get hand-me-down computers and disproportionately benchmark those to figure out what they're good for. You can invent a million scenarios, but it's extremely plausible that January results could be an outlier every year.
kazinator 2y ago
Sample size cannot compensate for sampling bias.

crazygringo 2y ago

> Their sample size has dropped dramatically since the previous year, from 100k to 25k for laptops and from 186k to 48k for desktops. I suspect that's just an effect of 2025 data being limited to just ~January, rather than a full 12 months. If people run a benchmark only once every 4 months on average, that would certainly explain the sample size.

sonofhans 2y ago
Yeah, good point. Or maybe it means people got slightly less-performant systems for Christmas than last year. Gates’ Law finally catching up with us all.
- deleted 2y ago
  [deleted]

veqq 2y ago

> “What Andy [Grove] giveth, Bill [Gates] taketh away.” Also known as https://en.wikipedia.org/wiki/Wirth%27s_law https://en.wikipedia.org/wiki/Wirth%27s_law

rcarmo 2y ago

I think there are several factors at play here, starting with “my current machine is good enough” and lower PC hardware sales as well as indeed, a bit of a shift to lower-power, quieter, smaller machines, and CPU characteristics changing (with more e-cores). Example: The mini-PC market has exploded, and people buying those are neither benchmark enthusiasts nor an active part of their readership.