zak

7 ms·

31.
by zak 7y ago
That's a great question. I don't have that analysis handy, but it would definitely be worth doing.
32.
by zak 7y ago
Thanks for the feedback. As more and more new ML system architectures emerge over the next few years, these performance comparisons will get even more complicated. Chip-to-chip comparisons and server-to-server comparisons are arbitrary, and
33.
by zak 7y ago
Author of the blog post here. We submitted multiple results using various Cloud TPU v3 Pod slice sizes to show the current achievable Transformer training efficiency at several scales: https://mlperf.org/training-results-0-6
34.
by zak 7y ago
Author of the blog post here. Cloud TPUs are designed to maximize performance-per-dollar, so you are right that pure performance comparisons at maximum scale don't tell the whole story. The most straightforward performance-per-dollar c
35.
by zak 8y ago
Actually, anyone can use a Cloud TPU v2 (one of the accelerators mentioned in the blog post) for free via Colab: https://colab.research.google.com/notebooks/tpu.ipynb Several other TPU-enabled Colabs are linked from th
36.
by zak 8y ago
At present, preemptible Cloud TPU v2 and v3 devices are widely available, and they are likely to be the most cost-effective option for training any of the models listed here, often by a wide margin: https://cloud.google.com&#x2F
37.
by zak 8y ago
+1 on open sourcing the code. If you post it somewhere, we'll take a look to see why the Adam optimizer isn't behaving as expected in your implementation. It may also be helpful to compare your code with the Fashion MNIST Colab ex
38.
by zak 8y ago
Here is a larger-scale comparison of Cloud TPU and Google Cloud GPU performance and cost (focused on Cloud TPU Pods): https://cloud.google.com/blog/products/ai-machine-learning/n... All the code used in that
39.
by zak 8y ago
We're on it! = D
40.
by zak 8y ago
You can also use Cloud TPUs for free in your browser via Colab. Several sample notebooks are available here: https://www.tensorflow.org/tfrc/