According to a Google blog post from 2016 [1], use-cases were RankBrain to improve the relevancy of search results and Street View. Also they used it for AlphaG

According to a Google blog post from 2016 [1], use-cases were RankBrain to improve the relevancy of search results and Street View. Also they used it for AlphaGo. And from what I remember from my MSc thesis, they also probably were starting to use it for Translate. I can't find any TPU reference in the Attention is All You Need or BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, but I have been fine-tuning BERT in a TPU at the time in okt 2018 [2]. If I remember correctly, the BERT example repository showed how to fit a model with a TPU inside a Colab. So I would guess that the natural language research was mostly not on TPU's around 2016-2018, but then moved over to TPU in production. I could be wrong though and dgacmu probably knows more. [1]: https://cloud.google.com/blog/products/ai-machine-learning/google-supercharges-machine-learning-tasks-with-custom-chip https://cloud.google.com/blog/products/ai-machine-learning/g... [2]: https://github.com/rikhuijzer/improv/blob/master/runs/2018-11-28%20askubuntu%20bert-base%2030%20epochs%20f1/GPU_Google_BERT_Sentence_Classification.ipynb https://github.com/rikhuijzer/improv/blob/master/runs/2018-1...