4 ms·
As an aside, Google used CPU-based machine learning (using enormous numbers of CPUs) for a long time before custom ASICS or tensorflow even existed. The big on
by dekhn 2y ago
As an aside, Google used CPU-based machine learning (using enormous numbers of CPUs) for a long time before custom ASICS or tensorflow even existed.
The big ones were SmartASS (ads serving) and Sibyl (everything else serving).
There was an internal debate over the value of GPUs with a prominent engineer writing an influential doc that caused Google continue with fat CPU nodes when it was clear that accelerators were a good alternative. This was around the time ImageNet blew up, and some eng were stuffing multiple GPUs in their dev boxes to demonstrate training speeds on tasks like voice recognition.
Sibyl was a heavy user of embeddings before there was any real custom ASIC support for that and there was an add-on for TPUs called barnacore to give limited embedding support (embeddings are very useful for maximizing profit through ranking).
- mmx1 2y agoWhat I've heard is that the extrapolation of compute needed so many additional CPU servers to keep running the existing workload types that it obviously justified dedicated hardware. Same for video encoding accelerators[1]. [1]: https://research.google/pubs/warehouse-scale-video-acceleration-co-design-and-deployment-in-the-wild/ https://research.google/pubs/warehouse-scale-video-accelerat...
- dekhn 2y agoyes, that was one of the externally shared narratives. The other part is that google didn't want to be beholden to nvidia GPUs, since they have an associated profit margin that is higher than TPUs, as well as resource constraints (total amount of GPUs shipping at any given time). Another part that was left out was that Google did not make truly high speed (low-latency) networking and so many of their CPU jobs had to be engineered around slow networks to maintain high utilization and training speed. Google basically ended up internally relearning the lessons that HPC and supercomputing communities had already established over decades.
- miki123211 2y ago> The big ones were SmartASS (ads serving) and Sibyl (everything else serving). Ah, the days when you, as a tech company employee, could call a service "SmartASS" and get away with it...