4 ms·

Google is certainly Nvidia's top customer.

• varelse 11y ago
  Sure, now. 4 years ago, not so much. One might even conclude that one person inside Google had written a brawny and influential paper that was right in a lot of ways, but wimpily dead-wrong to apply to GPUs.
  • ska 11y ago
    Might one conclude that getting hung up on one opinion from a large group of people projected forward in time is... not very useful?
    • varelse 11y ago
      Might one conclude that this influential opinion might have created a big old stinky career mess for someone in the field that took many months to clean up and cost one a fair chunk of change to do so? So perhaps one could be forgiven for noting such a disruptive change in viewpoint? Now we all must own our choices and one was very silly, stupid, and naive to blindly accept a position at Google, but everyone makes mistakes, no? Further, TensorFlow looks fantastic but given that even just a year ago, when one was recruited to return to Google for !(Deep Learning) but for something which also happened to be another braindead obvious choice to run on GPUs, one was informed said viewpoint on GPUs was still in effect and declined. So what I'd really love to know is how the message finally sunk in? I'm betting there's a really great story here.
      • chubot 11y ago
        Uh, so what happened? Were you working on a GPU project at Google before management came around to this way of thinking, and it got canned? That sucks, but that kind of thing happens a lot. It's not that surprising. Google wanted a homogeneous infrastructure for a long time. But a new application (neural networks) motivated a new infrastructure (GPUs in clusters). There was that Stanford paper everyone talks about which compared training a neural network on Google's 16K cores (DistBelief paper) vs a handful of GPUs with infiniband. Even Andrew Ng has seemed to subtly criticize Google for thinking "the old way" (i.e. "cloud" technology vs HPC technology, HPC being more effective for neural nets) Also, I don't quite understand your comment about brawny vs. wimpy. Yes Urs wrote a rebuttal against wimpy cores. As far as I remember, his argument is basically that there is some portion of software on the critical latency path that's not multithreaded. wimpy cores make the most sense in a world of perfect parallelism.
        • varelse 11y ago
          I'm done ranting about Google and GPUs at last. From a technical standpoint, I believe TensorFlow vindicates my viewpoint and that's enough for me. This is closure I really needed, OK? And besides, now I can concentrate my energy on asking WTH happened to Android performance between 4.x and 5.x J/K? And from a professional standpoint, I moved on and fixed the damage I let them inflict on my career. I eventually built exactly what I was told couldn't and shouldn't be done with GPUs. But back then, the argument Google made to me was that too much of the workloads I wanted to run on GPUs were serialized (hence brawny versus wimpy). And from a classical parallel algorithms perspective, they were correct. The problem is that from a GPU programming perspective, they weren't even wrong(tm). And at the time, I had ~5 years of CUDA programming (very early adopter) and 30,000+ lines of CUDA code to back that up whereas the people I tried to convince had zero such experience so there was just no way. My argument boiled down to TLDR: O(n^2) or higher probably belongs on GPUs. Would anyone disagree these days?
          • lmeyerov 11y ago
            More economic argument: any sufficiently restricted and important algorithm will fund enough (1) PhDs to pull out the parallelism and then (2) devoted engineers to map it down successively close to the metal (perf/watt), even if the legacy code is polished. CPU->GPU->FPGA->ASICS. Regex, neural nets, doesn't really matter.