4 ms·

As the chart is updated bi-weekly, but the data point that may change is for the current year. The first few days or weeks of a new year are less accurate compa

As the chart is updated bi-weekly, but the data point that may change is for the current year. The first few days or weeks of a new year are less accurate compared to the end of a year.

moffkalast 2y ago
January 2024: https://web.archive.org/web/20240123174954/https://www.cpubenchmark.net/year-on-year.html https://web.archive.org/web/20240123174954/https://www.cpube... January 2023: https://web.archive.org/web/20230130185431/https://www.cpubenchmark.net/year-on-year.html https://web.archive.org/web/20230130185431/https://www.cpube... Still, both previous years were notably up after the first month already. This is different and perhaps notable regardless. Either the release schedule is skewed forward or the hardware is genuinely stagnating. Or perhaps the benchmark is hitting some other CPU unrelated bottleneck.
- palijer 2y ago
  This is just still a sample size of three for a data period that is stated to be less accurate" not "has lower values" Just because the error isn't presented the same here doesn't mean it's error free...
- lolinder 2y ago
  http://web.archive.org/web/20240221001449/https://www.cpubenchmark.net/year-on-year.html http://web.archive.org/web/20240221001449/https://www.cpuben... They have more than twice as many samples on their Feb 10 update this year (47810) as they did last year on Feb 14 (22761). They have shown some growth year on year in sample size, but nowhere near doubling. That suggests this month has been an outlier in having a strangely large number of samples already, which could all be related—maybe their software started being bundled with a specific OEM or was featured on a popular content creator or whatever. As a sibling notes, less accurate just means less accurate, not necessarily skewed upward. There simply is less data to draw conclusions on than there will be later, so any medium-sized effect that adds 20k extra samples will have a larger effect now than later.
  - hsuduebc2 2y ago
    I had the same thought. The common reasons for such a decline would be something like change of CPU architecture prioritizing energy efficiency over raw performance, limitations in manufacturing processes, or changes in benchmarking methodologies wouldn't had such steep decline. So I would guess either change of methodology or as you said weaker CPUs in general are measured by that.
  - moffkalast 2y ago
    So if I understand right, more old hardware makes it into the sample now than before, increasing the unreliability of early data? That makes sense I guess.
    - lolinder 2y ago
      Not increasing the unreliability—it's unreliable no matter what this early in the year—but it's possible that more old hardware made it in this past month than in previous early years which would explain why the number went down this time.
ellisv 2y ago
Really makes you wish the chart showed some measure of variance
- throwaway287391 2y ago
  Or just have the last data point include everything from the full 12 month period before (as the title "year on year" would suggest) and maybe even put it in the correct place on the x-axis (e.g. for today, Feb 12, 2025, about 11.8% of the full year gap width from the (Dec 31) 2024 point).
  - contravariant 2y ago
    And then write an article every month about how the year on year difference has(n't) gone up/down yet.
- deleted 2y ago
  [deleted]
- kqr 2y ago
  We still know the current data point has about three times the standard error of the previous point, but I agree it's hard to say anything useful without knowledge of the within-point variance.
- SubiculumCode 2y ago
  variance. Have you noticed that the mainstream public-level discussion of almost any topic never progresses farther than a point estimate? Variance implies nuance, and nuance is annoying to those who'd just rather paint a story. Variance isn't even that much nuance, because it is also just a point estimate for the variability of a distribution, not even its shape. Public discourse is stuck at the mean point estimate, and as such, is constantly misled. This is all an analogy, but feels very true.
  - contravariant 2y ago
    Sometimes even an average is too much to ask. One of my pet peeves is articles about "number changed". Averages and significance never enter the discussion. Worst are the ones where he number is the number of times some random event happened. In that case there's a decent chance the difference is less than twice the square root, so assuming a Poisson distribution you know the difference is insignificant.
TheSpiceIsLife 2y ago
So they have a dataset, and they’ve cherry picket a time period to time period, and not done any location based sorting etc, to suit a narrative. Whatever pays the bill I suppose.

As the chart is updated bi-weekly, but the data point that may change is for the current year. The first few days or weeks of a new year are less accurate compared to the end of a year.

moffkalast 2y ago

January 2024: https://web.archive.org/web/20240123174954/https://www.cpubenchmark.net/year-on-year.html https://web.archive.org/web/20240123174954/https://www.cpube... January 2023: https://web.archive.org/web/20230130185431/https://www.cpubenchmark.net/year-on-year.html https://web.archive.org/web/20230130185431/https://www.cpube... Still, both previous years were notably up after the first month already. This is different and perhaps notable regardless. Either the release schedule is skewed forward or the hardware is genuinely stagnating. Or perhaps the benchmark is hitting some other CPU unrelated bottleneck.

palijer 2y ago
This is just still a sample size of three for a data period that is stated to be less accurate" not "has lower values" Just because the error isn't presented the same here doesn't mean it's error free...
lolinder 2y ago
http://web.archive.org/web/20240221001449/https://www.cpubenchmark.net/year-on-year.html http://web.archive.org/web/20240221001449/https://www.cpuben... They have more than twice as many samples on their Feb 10 update this year (47810) as they did last year on Feb 14 (22761). They have shown some growth year on year in sample size, but nowhere near doubling. That suggests this month has been an outlier in having a strangely large number of samples already, which could all be related—maybe their software started being bundled with a specific OEM or was featured on a popular content creator or whatever. As a sibling notes, less accurate just means less accurate, not necessarily skewed upward. There simply is less data to draw conclusions on than there will be later, so any medium-sized effect that adds 20k extra samples will have a larger effect now than later.
- hsuduebc2 2y ago
  I had the same thought. The common reasons for such a decline would be something like change of CPU architecture prioritizing energy efficiency over raw performance, limitations in manufacturing processes, or changes in benchmarking methodologies wouldn't had such steep decline. So I would guess either change of methodology or as you said weaker CPUs in general are measured by that.
- moffkalast 2y ago
  So if I understand right, more old hardware makes it into the sample now than before, increasing the unreliability of early data? That makes sense I guess.
  - lolinder 2y ago
    Not increasing the unreliability—it's unreliable no matter what this early in the year—but it's possible that more old hardware made it in this past month than in previous early years which would explain why the number went down this time.

ellisv 2y ago

Really makes you wish the chart showed some measure of variance

throwaway287391 2y ago
Or just have the last data point include everything from the full 12 month period before (as the title "year on year" would suggest) and maybe even put it in the correct place on the x-axis (e.g. for today, Feb 12, 2025, about 11.8% of the full year gap width from the (Dec 31) 2024 point).
- contravariant 2y ago
  And then write an article every month about how the year on year difference has(n't) gone up/down yet.
deleted 2y ago
[deleted]
kqr 2y ago
We still know the current data point has about three times the standard error of the previous point, but I agree it's hard to say anything useful without knowledge of the within-point variance.
SubiculumCode 2y ago
variance. Have you noticed that the mainstream public-level discussion of almost any topic never progresses farther than a point estimate? Variance implies nuance, and nuance is annoying to those who'd just rather paint a story. Variance isn't even that much nuance, because it is also just a point estimate for the variability of a distribution, not even its shape. Public discourse is stuck at the mean point estimate, and as such, is constantly misled. This is all an analogy, but feels very true.
- contravariant 2y ago
  Sometimes even an average is too much to ask. One of my pet peeves is articles about "number changed". Averages and significance never enter the discussion. Worst are the ones where he number is the number of times some random event happened. In that case there's a decent chance the difference is less than twice the square root, so assuming a Poisson distribution you know the difference is insignificant.

TheSpiceIsLife 2y ago

So they have a dataset, and they’ve cherry picket a time period to time period, and not done any location based sorting etc, to suit a narrative. Whatever pays the bill I suppose.