5 ms·
And just 4 years after their insistence that GPUs would never have a role at Google... How 'bout that?
by varelse 11y ago
And just 4 years after their insistence that GPUs would never have a role at Google... How 'bout that?
- pacala 11y agoGoogle is a big place, with diverse opinions. Even better, it encourages updating one's opinion in face of new evidence. GPUs have proven themselves cost-effective in tackling a number of vision problems [and others], Google has those problems, so the opinion was updated and the problems were solved in a cost-effective manner.
- varelse 11y agoThat... Was not... My experience there... But kudos to the deep learning guys for overcoming that potential energy barrier NVIDIA couldn't surmount on their own...
- Florin_Andrei 11y agoTo me, processing lots and lots of data with "relatively simple" algorithms spells GPU, pretty much. Machine learning seemed destined to bump into GPUs sooner or later.
- varelse 11y agoWhy yes... https://github.com/BIDData/BIDMach https://github.com/BIDData/BIDMach
- pilooch 11y agoGoogle is certainly Nvidia's top customer.
- varelse 11y agoSure, now. 4 years ago, not so much. One might even conclude that one person inside Google had written a brawny and influential paper that was right in a lot of ways, but wimpily dead-wrong to apply to GPUs.
- ska 11y agoMight one conclude that getting hung up on one opinion from a large group of people projected forward in time is... not very useful?
- varelse 11y agoMight one conclude that this influential opinion might have created a big old stinky career mess for someone in the field that took many months to clean up and cost one a fair chunk of change to do so? So perhaps one could be forgiven for noting such a disruptive change in viewpoint? Now we all must own our choices and one was very silly, stupid, and naive to blindly accept a position at Google, but everyone makes mistakes, no? Further, TensorFlow looks fantastic but given that even just a year ago, when one was recruited to return to Google for !(Deep Learning) but for something which also happened to be another braindead obvious choice to run on GPUs, one was informed said viewpoint on GPUs was still in effect and declined. So what I'd really love to know is how the message finally sunk in? I'm betting there's a really great story here.
- chubot 11y agoUh, so what happened? Were you working on a GPU project at Google before management came around to this way of thinking, and it got canned? That sucks, but that kind of thing happens a lot. It's not that surprising. Google wanted a homogeneous infrastructure for a long time. But a new application (neural networks) motivated a new infrastructure (GPUs in clusters). There was that Stanford paper everyone talks about which compared training a neural network on Google's 16K cores (DistBelief paper) vs a handful of GPUs with infiniband. Even Andrew Ng has seemed to subtly criticize Google for thinking "the old way" (i.e. "cloud" technology vs HPC technology, HPC being more effective for neural nets) Also, I don't quite understand your comment about brawny vs. wimpy. Yes Urs wrote a rebuttal against wimpy cores. As far as I remember, his argument is basically that there is some portion of software on the critical latency path that's not multithreaded. wimpy cores make the most sense in a world of perfect parallelism.
- 11y ago