8 ms·
Microsoft Incorporates Graphcore AI Chips in Azure Cloud
- orf 7y agoInteresting, in their benchmarks it shows that Tensorflow is nearly twice as slow to train on a GPU compared to PyTorch. That's a surprisingly large difference.
- duaoebg 7y agoThe Tensorflow numbers are estimated. They may have estimated wrong.
- _coveredInBees 7y ago2X slower is unlikely, but Pytorch does tend to be faster than TF in most recent benchmarks. See this link posted by another commenter for one example: https://wrosinski.github.io/deep-learning-speed-vol1/ https://wrosinski.github.io/deep-learning-speed-vol1/
- fxtentacle 7y agoThe author seems to be completely clueless. 1. "This is the first time any major cloud service provider has publicly offered customers the opportunity to run their data on an accelerator from any of the dozens of AI chip startups" No. Google purchased an AI chip startup around 2015 and since February 2018 you could use TPUs on their cloud. The TPU is a hardware accelerator for AI and matrix multiplication. 2. The TensorFlow values in their diagram are "* estimated" which I guess makes sense because TensorFlow & TPU is the biggest competitor to Graphcore. Thanks to XLA, TF2 tends to be a few percent faster on GPU than PyTorch. For example, see here: https://wrosinski.github.io/deep-learning-frameworks/ https://wrosinski.github.io/deep-learning-frameworks/ 3. They compare Graphcore against GPU, but the real comparison target here would be other accelerators, e.g. TPU. Graphcore comes out similarly fast as a V100, meaning it would be roughly 50% slower than a TPU.
- justicezyx 7y ago> 3. They compare Graphcore against GPU, but the real comparison target here would be other accelerators, e.g. TPU. Graphcore comes out similarly fast as a V100, meaning it would be roughly 50% slower than a TPU. Hmm, business wise, GPU is every AI accelerator startups biggest enemy. BTW we are building software infrastructure for AI chips, ping me if interested. info@nascentcore.com
- deepnotderp 7y agoI'm interested, but you don't seem to have an email address I can ping you at.
- justicezyx 7y agoAdded an email.
- KidComputer 7y agoThose benchmarks are ridiculously out of date. See https://wrosinski.github.io/deep-learning-speed-vol1/ https://wrosinski.github.io/deep-learning-speed-vol1/ for updated benchmarks by the same person. PyTorch beats Tensorflow on every single model.
- duaoebg 7y agoTraining ResNet with 8IPUs is roughly the same as 8GPUs. I doubt it was Tensor Core optimized so I assume the GPUs can go way faster. I'd love to spend some time and come up with Graphcore optimized models. But at this stage fused Tensor Core optimized ops are much more interesting to me.
- justicezyx 7y agoYep, nVidia CEO claimed nVidia is really a software company citing the complex software they built on top of their GPU. But I think AI as a whole needs to make optimizing ML AI workloads on non GPU accelerators equally well. Otherwise the industry will be stifled by the money seeking shortsigtness of nVidia
- sbierwagen 7y agoEasy enough to optimize a net against a new architecture: https://ai.googleblog.com/2019/08/efficientnet-edgetpu-creating.html https://ai.googleblog.com/2019/08/efficientnet-edgetpu-creat...
- justicezyx 7y agoSeems this is to using automl to find NN architecture that fits well in certain inference chips. The orthogonal issue is given a trained model how to make it run faster on a given inference chip. Either way, a blog post on Google AI is a symbol that 1% of the work is done, the rest 99% needs more investment from the believers.
- mr_toad 7y agoThe architecture of these seems quite different from GPUs and Google’s TPUs. It’s interesting that they went with smaller amounts of on-core memory and very high bandwidth, rather than a large amount of on-card memory.
- Abishek_Muthian 7y agoI'm afraid these AI accelerators available only to the Big cloud companies would further 'prevent' democratisation of AI/ML. GPU is already a big bottleneck for an individual/startup from a poor economic region. Considering, even entry level Software Engineering jobs now expect the candidates to know ML; it is imposing a huge disadvantage over a large sector of students from poor countries. I understand this issue has largely to do with how semiconductor industry by itself is structured. With only handful of fabrication plants capable of mass producing semiconductors of this nature and they being integral part of 'soft power' in Geo-politics; it's hard for a startup to enter this space and when they do, they have no option to tie-up with these cloud companies. But, without the hardware bottleneck being addressed not only 'AI powered end product' will increase the in-equality; but the ML education/research ecosystem is already raising the inequality.
- sanxiyn 7y agoNote that in this case, exactly same accelerators will be available as PCIe cards attached to Dell machines. So none of your concerns apply.
- Abishek_Muthian 7y agoAvailability of Graphicscore as PCIe only on Dell machines will be more inaccessible than expensive GPUs. Only mass market open hardware based on RISC-V or at-least affordable ARM based hardware like Jetson Nano but with actual training capabilities can address this issue. To put it blunt, we need openAI for hardware & not just software stack.
- nl 7y agoNvidia GTX 1060s are available for less than a Jetson and train small models fairly well, and fine tune large models very well.
- Abishek_Muthian 7y agoNot in developing nations, card itself is twice that of Nano, total PC cost/power profile would be much more expensive.
- mhishambr 7y agoThat is a very good development