4 ms·
I'm done ranting about Google and GPUs at last. From a technical standpoint, I believe TensorFlow vindicates my viewpoint and that's enough for me. This is cl
by varelse 11y ago
I'm done ranting about Google and GPUs at last. From a technical standpoint, I believe TensorFlow vindicates my viewpoint and that's enough for me. This is closure I really needed, OK? And besides, now I can concentrate my energy on asking WTH happened to Android performance between 4.x and 5.x J/K?
And from a professional standpoint, I moved on and fixed the damage I let them inflict on my career. I eventually built exactly what I was told couldn't and shouldn't be done with GPUs.
But back then, the argument Google made to me was that too much of the workloads I wanted to run on GPUs were serialized (hence brawny versus wimpy). And from a classical parallel algorithms perspective, they were correct. The problem is that from a GPU programming perspective, they weren't even wrong(tm). And at the time, I had ~5 years of CUDA programming (very early adopter) and 30,000+ lines of CUDA code to back that up whereas the people I tried to convince had zero such experience so there was just no way. My argument boiled down to TLDR: O(n^2) or higher probably belongs on GPUs. Would anyone disagree these days?
- lmeyerov 11y agoMore economic argument: any sufficiently restricted and important algorithm will fund enough (1) PhDs to pull out the parallelism and then (2) devoted engineers to map it down successively close to the metal (perf/watt), even if the legacy code is polished. CPU->GPU->FPGA->ASICS. Regex, neural nets, doesn't really matter.