9 ms·
A few of the basic concepts: Smarter memory design means that data does not need to be constantly reloaded, saving time that would otherwise be wasted on data
by bmh100 10y ago
A few of the basic concepts:
Smarter memory design means that data does not need to be constantly reloaded, saving time that would otherwise be wasted on data transfer.
Using lower-precision, fixed-point arithmetic is more efficient than standard-precision floating-point arithmetic. As a result, less time and components are wasted on unnecessary precision.
Someone more knowledgeable about processors can better describe the benefits of the Wave architecture.
In terms of benchmarks, Wave claims[1] that a single Wave machine is 25x more efficient than 100 GPUs for Google's Inception network.
[1]: http://wavecomp.com/technology/ http://wavecomp.com/technology/
- varelse 10y ago100 of which GPU? And how were they connected to each other? They almost never give enough details to really understand these bold claims. If this is 100 K40s in a 10 Gb/s datacenter, this is old hat. If this is 100 Titan XPs in Big Sur boxes connected by 100+ Gb/s Infiniband, it's really interesting. I doubt it though. I suspect the only immediate threat to NVIDIA is underestimating AMD GPUs. 2+ years out, who knows?
- bmh100 10y agoWave seems to be referencing this[1] Google Research blog post. I couldn't find any reference to what GPU cards Google uses. It still provides a basis of comparison to Google Cloud at least. But like you say, a specialized data center deployment probably wouldn't show such a dramatic difference. [1]: https://research.googleblog.com/2016/04/announcing-tensorflow-08-now-with.html https://research.googleblog.com/2016/04/announcing-tensorflo...