9 ms·

Slight meta, and part of a larger 'rant,' but when are we going to get away from reviews that might as well have been printed on glossy pages in PC Magazine in

• localhost 7y ago
  Jupyter could form the basis of an interesting publishing pipeline for this kind of a review using various Python viz libraries. For user-contributed content, it would be interesting if you could "fork" an existing publication and add your own color to it. But somebody would need to pay for the compute and storage for this to work, as well as enable the kind of advertising / layout requirements that drive the business side of sites like Anandtech.
• lettergram 7y ago
  Not trying to be snarky, but: If you think that it’s be better - why don’t you make it? I suspect the current format is the way it is because it’s easier and “good enough”. I feel if you reach out to Phoronix you might get a response.
  • davidy123 7y ago
    > If you think that it’s be better - why don’t you make it? Because I am already quite busy on other projects.
• whatshisface 7y ago
  Dynamic JS graphs sound like a pain, I'm not a professional user of this specific website, and I don't want to learn a new dynamic interface for every website I visit. Sometimes I just want to look at a chart. User-contributed data would be of limited value on a website whose specific value proposition is that they know more about benchmarking than the average Joe. The pagination is annoying, though.
  • driverdan 7y ago
    It's really not. Generating them involves passing a collection of data to a script. Images can be generated server side for use by people who disable JS.
• dageshi 7y ago
  They don't do it because it won't lead to any additional page views but will require extra effort.
  • davidy123 7y ago
    This is probably the real answer. No profit innit. Having been through a few revolutions, there are advantages and challenges we won't know until we get there. Open it up so the users are encouraged to participate, rather than just using their computers to decide what new gaming system to build, with "participation" a ten comment per page system designed to support throwaways.
• pg_is_a_butt 7y ago
  Is the larger "rant" that you are funded by Intel, and their market dominance is drying up, and you find yourself attacking reporters as a last ditch effort?
• bluegreyred 7y ago
  the german site computerbase.de does this. they also do crowdsourced benchmarks for various applications and video game titles. I find that their articles are some of the best on the web, though you'll need machine translation to read them. case in point: https://www.computerbase.de/2019-11/amd-ryzen-3950x-test/3/#abschnitt_benchmarks_in_anwendungen https://www.computerbase.de/2019-11/amd-ryzen-3950x-test/3/#... you can press the button on the top right of the graph to show/hide other, less relevant entries. click an entry to lock it in for relative comparison. the top dropdown menu lets you choose between multicore, singlecore and application specific results.
  • davidy123 7y ago
    It's funny how the German sites tend to be so much better at this. I don't follow computerbase, but notebookcheck has some of the most thorough and consistent reviews, and they allow cross-device comparison. They cover news now, too, though I consider that a drawback since it's less focused. There are a very large number of review sites, Anandtech being one of the first, presumably all following the same sustainability formula, but despite all that effort, consistency, thoroughness, built in tools, and building on aggregated output are the exception.
• joaobeno 7y ago
  There are huge selections of manufactures, speeds, powers, capacities, and so on... Thus, it is hard to simply let you pick and choose... They pick some parts, limit external factors that could influence the benchmarks, and show you the metrics so you can make an somewhat more informed decision...
• bransonf 7y ago
  I mean, there are plenty of options to compare new cpus to old ones. 0. CPUBoss 1. CPUBenchmark 2. UserBenchmark And if you’re considering an upgrade, it’s almost always the case that you should be comparing latest generation offerings to arrive at a purchase decision. Don’t call the web stagnant because they decide not to flood the page with JS libraries.
  • ihattendorf 7y ago
    But if you're deciding whether or not to upgrade, a comparison between your current hardware and newer hardware is relevant.
    • bransonf 7y ago
      That’s why I suggested the first three websites. The role of Anandtech/TomsHardware and the like is not to do this. It’s to give news about new CPUs and where the fit in current offerings. They are news sites after all.
  • solotronics 7y ago
    CPUBoss sucks. It doesnt have chips from the past 4 years in it.
  • IanCutress 7y ago
    www.anandtech.com/Bench for real world benchmark data
• uluyol 7y ago
  Few figures benefit from being interactive. I'm confused as to why why you'd want user contributed results. Getting a benchmark setup to be fair takes effort and you cannot simply take numbers measured using different CPU/memory/OS version/etc combos and draw conclusions from them. The fact that components combos are curated is a feature. Personally, all I want from Anandtech is better editing and more articles.
• tedunangst 7y ago
  https://www.anandtech.com/bench/product/2538?vs=2558 https://www.anandtech.com/bench/product/2538?vs=2558
  • deleted 7y ago
    [deleted]
• wtallis 7y ago
  > Instead we get feeble excuses like "it doesn't make sense to compare a two-year-old-generation to this one," well yes it does if I'm considering an upgrade. You don't get that excuse from AnandTech. We do our best to keep a long history of benchmark data for users to peruse: https://www.anandtech.com/bench/CPU-2019/2224 https://www.anandtech.com/bench/CPU-2019/2224 The main limiting factor on how far back our benchmark database goes is software updates. When we have to update the OS or CPU microcode for Spectre, Meltdown, etc., or update GPU drivers, that invalidates results, and re-testing a large pile of older hardware takes a long time. Historically this has mostly been a problem for GPUs since their drivers are such a moving target, but the past two years of CPU vulnerabilities have been a hassle.
  • davidy123 7y ago
    That's great to see, thanks. I thought I had asked about it some time ago to no avail. Still, if the interest is moving the front forward, making it available as cc-by would be nice, so people could use and contribute their own analysis around it.
• xondono 7y ago
  User contributed is not a good idea. One of the sad realities is that even in the enthusiast market a lot of people don’t know/don’t care to tune and test their system, and there’s a lot of traps that would make the results useless. There’s some of it that you can check and verify, but things like watercooling or chosing the appropriate RAM configuration and timing will have a big impact.
  • davidy123 7y ago
    I think this is solve-able through user reputation.
    • mmmrk 7y ago
      Reputation is not an objective measure.
      • davidy123 7y ago
        It's not, but it's a measure, like money, that can help sort high quality/relevant contributions. Phoronix / OpenBenchmarking.org provides an example of how many technical people can provide benchmarks for many component and software combinations, in the mainstream reputation would help for even broader views.
• puranjay 7y ago
  On the content side, it isn't financially beneficial to invest in higher quality content or presentation. Most websites that aren't outsourcing their content to $10 writers on UpWork are already running on razor thin margins. Outside of a handful of big name publishers (the NYTs and Economists of the world), paid subscriptions are unviable for most. The lack of innovation on the web can be directly attributed to the lack of advertising money on the web - outside of Google and FB, of course.
• yifanlu 7y ago
  > and user contributed There's been a lot of debate in the enthusiasts community, but reviewers believe that user benchmarks don't have much value. There's so much variation in software, cooling, RAM speed, GPU speed, etc. Even misconfigurations like different running background apps can skew the results. However, for a processor that's been in the market for a while, I think userbenchmarks is a good site to look at the aggregate data. Their rankings were recently updated to disfavor AMD chips, so don't take those too seriously. But for head-to-head comparisons of processors with a lot of users, you can get a good idea of how much faster a processor is. However, I disagree that review sites should consider "user data" because 1) these are new processors and people who read these reviews are usually early adopters who want to make a buying decision and 2) the testing setup and methodology is a time consuming and scientific process and shouldn't be discounted by just asking random people to run an app.
• Dunedan 7y ago
  > Anandtech has been at it since the 90s and hasn't changed their format at all. This is a serious decades-long stagnation of the web. Aside from the quality of their content, their format is something I really like about Anandtech. Is it really stagnation when you have figured out a format which works and stick to it?
• xbkingx 7y ago
  I had a side job as a tech reviewer ~15 years ago and I can tell you why this is never going to happen. It's a classic case of "works in theory, but impossible in practice." What you're actually asking for is a database of every permutation of hardware, hardware revision, firmware/BIOS, benchmarking software, benchmarking software version, driver version, OS, OS patch, user selectable feature state, etc. is tracked and tested. It's a nearly infinite data set size, and you'll still miss things like ambient room temperature/humidity for CPU thottling, that one finicky USB device you spilled water on that sometimes disconnects randomly, and specific workflow quirks. "Objective" benchmarks were an almost tractable problem 15-20 years ago, but with the way modern OSes run background tasks, access network resources, and perform self-maintenance, it's even more difficult to bridge the synthetic-to-real world divide. So, you select a point of reference (e.g. - new GPU), you choose the most common system components and a selection of components that tell a story (usually "if you're building a new PC now, here are the options"), you assemble the current versions of all software/drivers, and run your tests x times. The good sites will have "sub-stories," like specific workflows or new/changed features, but once you get to around 4 of these, you start hitting too many permutations to clearly communicate the significance of your choices. Even if you crowdsourced it, it's A) extremely difficult to verify the integrity of results and prevent manipulation from marketing departments or brigading, and B) an unrewarding, tedious process where the best practice is to leave the machine untouched for hours. Most of the crowdsourced benchmarking sites are set up to be little competitions and sanity checks when overclocking. For a pure review of shipping hardware/software, you run the benchmark and you're done. It isn't particularly fun, requires invasive cataloguing of system specs, and isn't very new-user/first time builder friendly. Dynamic graphs would be nice (I love them), but I suspect many review sites have run the numbers and found they reduce overall web metrics. Many (I would argue most) people don't notice interactive page components, aren't interested enough to turn their quick article skim into a deep dive, or are reading in less optimal settings (e.g. - on phone on the toilet). Instead of getting 7 page views for 1 minute each with separate graphs, you're getting 1 page view for either 1 minute (probably 70%) or 10 minutes (probably 30%) with a dynamic graph. I'm a total data junkie and I WISH there was a good solution to these problems, but there isn't a practical implementation that I've seen or dreamt up that doesn't have enough variance to render the point of having such fine-grained data moot.