3 ms·
Given the impressive inference results shown on this page, I find it interesting that graphcore did not participate in mlperf inference v0.5.
by rrss 7y ago
Given the impressive inference results shown on this page, I find it interesting that graphcore did not participate in mlperf inference v0.5.
- m0zg 7y agoIt's probably because these are the best hand picked results they could come up with (and also, crucially, _worst_ for the competition), and mlperf has a fixed set of tasks and you don't get to do that. I've done this shit professionally: if you're given full freedom wrt the choice of the task and configuration, you can easily make it seem like your competition sucks beyond belief. You can "prove" that a CPU is faster than a GPU without much trouble. :-) And yet NVIDIA still dominates most rankings, mostly, even though in absolute terms it's nowhere near the fastest or the most efficient option (that'd be the TPU, on some tasks, at the moment). With so many performance cliffs in the hardware tooling is super important, and NVIDIA is the only viable acceleration option that has any meaningful tooling at all. Caveat emptor: the published numbers (_any_ numbers, not just Graphcore's) are mostly bullshit unless code is also published and hardware is available for independent measurement. There's no way you're getting the claimed 13TFLOPs out of your shiny new NVIDIA GPU. Take an off the shelf resnet50 (4GFLOPS) and witness it run at about 700 samples per second, which pencils out to 2.8TFLOPs not 13. Still amazing (and still easily 10x the high end CPU throughput), but perf claims are often exaggerated by as much as 10x even by big names, let alone startups.
- typon 7y agoThis is probably why they didn't publish any Resnet50 numbers in this post.