3 ms·
I’m almost certain that Google/Amazon/Apple et al have similar specialized computing hardware, but at smaller scales. We don’t publicly know much about their in
by extheat 4y ago
I’m almost certain that Google/Amazon/Apple et al have similar specialized computing hardware, but at smaller scales. We don’t publicly know much about their internal hardware. If they had the infra, I think it would be interesting to see some Tesla cloud service offering indeed.
[1] https://cloud.google.com/blog/products/ai-machine-learning/google-breaks-ai-performance-records-in-mlperf-with-worlds-fastest-training-supercomputer https://cloud.google.com/blog/products/ai-machine-learning/g...
- m00x 4y agoGoogle has had TPUs for a while. Most of the incredibly powerful transformer architectures like Parti and PaLM are trained on it. They even merge TPU pods with Pathways so they can go up to 540B parameters (gpt-3 is only 175B) https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture/ https://blog.google/technology/ai/introducing-pathways-next-...