3 ms·

The blind tests show something useful - consistency of throughput, which people notice, regardless of how benchmarks work. Benchmarking software should do more

by yarg 9y ago

The blind tests show something useful - consistency of throughput, which people notice, regardless of how benchmarks work. Benchmarking software should do more to provide a clean and meaningful representation of the distribution of performance - people care more about a card's worst case than its best or average. Another significant consideration - as alluded to in the TensorFlow comment - is the compute performance. Although compute performance is not yet that important from a gaming perspective, I would be rather surprised not to see some games start to utilise it for engine functionality - especially after Vulcan and OpenCL merge. Wait to see the performance of the card as a whole, and consider the implications for the next generation before you declare the card a non-starter and evolutionary dead-end.

TheGuyWhoCodes 9y ago
1) Blind Test - That's true, but still people who buy check benchmarks, don't forget AMD did exactly that with Bulldozer and we know how that ended up.... If AMD thought that they had a winning horse they would not hide the performance numbers, they would PR that like crazy, like they are doing with Ryzen and ThreadRipper. 2) I'm all for AMD to kick Nvidia's ass in compute and end cuds's reign but it won't sell gaming cards. Vega FE (A 1000$ card) is out for a month and I haven't seen anyone benchmark it and show that it's better, Even if games will use it there is a heck of a difference between building the model and running inference.
- yarg 9y ago
  It won't sell gaming cards yet. My argument was that comparing underutilised hardware to that with fundamental unresolvable physical bottlenecks seemed to be a little premature. The next generation of games could potentially utilise compute functionality, especially if Microsoft and Sony get off their arses and push the capability on consoles. (That said, porting such software to a PC environment would not be withouut significant difficulty, given the vastly different memory configuration.) I'll be hugely disappointed in the industry if we don't see some decent Vulcan based game engines in the next few years.
  - mtanski 9y ago
    consoles tend to have lower level hardware access then opengl or even dx12 and vulkan
gabriel34 9y ago
Benchmarking software should do more to provide a clean and meaningful representation of the distribution of performance - people care more about a card's worst case than its best or average. You should check Tech Report's Frame Time Analysis: http://techreport.com/review/31546/where-minimum-fps-figures-mislead-frame-time-analysis-shines http://techreport.com/review/31546/where-minimum-fps-figures...
- yarg 9y ago
  The single frame that takes the longest time to render won't normally be an issue, but a cluster of frames each taking significantly long than usual will result in a noticable stutter. The stats should try to take into account not only the low frame-rate but whether that rate has been held long enough to peturb a human observer.

by yarg 9y ago

TheGuyWhoCodes 9y ago

1) Blind Test - That's true, but still people who buy check benchmarks, don't forget AMD did exactly that with Bulldozer and we know how that ended up.... If AMD thought that they had a winning horse they would not hide the performance numbers, they would PR that like crazy, like they are doing with Ryzen and ThreadRipper. 2) I'm all for AMD to kick Nvidia's ass in compute and end cuds's reign but it won't sell gaming cards. Vega FE (A 1000$ card) is out for a month and I haven't seen anyone benchmark it and show that it's better, Even if games will use it there is a heck of a difference between building the model and running inference.

yarg 9y ago
It won't sell gaming cards yet. My argument was that comparing underutilised hardware to that with fundamental unresolvable physical bottlenecks seemed to be a little premature. The next generation of games could potentially utilise compute functionality, especially if Microsoft and Sony get off their arses and push the capability on consoles. (That said, porting such software to a PC environment would not be withouut significant difficulty, given the vastly different memory configuration.) I'll be hugely disappointed in the industry if we don't see some decent Vulcan based game engines in the next few years.
- mtanski 9y ago
  consoles tend to have lower level hardware access then opengl or even dx12 and vulkan

gabriel34 9y ago

Benchmarking software should do more to provide a clean and meaningful representation of the distribution of performance - people care more about a card's worst case than its best or average. You should check Tech Report's Frame Time Analysis: http://techreport.com/review/31546/where-minimum-fps-figures-mislead-frame-time-analysis-shines http://techreport.com/review/31546/where-minimum-fps-figures...

yarg 9y ago
The single frame that takes the longest time to render won't normally be an issue, but a cluster of frames each taking significantly long than usual will result in a noticable stutter. The stats should try to take into account not only the low frame-rate but whether that rate has been held long enough to peturb a human observer.