4 ms·
This manifests in the training time + cost for imagenet in the stanford DAWN benchmark. The (partial) TPU pods win on training time, with a clear exponential sp
by sdenton4 8y ago
This manifests in the training time + cost for imagenet in the stanford DAWN benchmark. The (partial) TPU pods win on training time, with a clear exponential speedup as the pod fraction increases. (How do you use more cores? Increase the batch size.)
https://dawn.cs.stanford.edu//2018/04/30/dawnbench-v1-results/ https://dawn.cs.stanford.edu//2018/04/30/dawnbench-v1-result...
- p1esk 8y agoI’m looking at Imagenet training time for Resnet-50 [1] and it appears that 32 V100 chips are faster than half of a TPUv2 pod (64 chips). Are we looking at different benchmarks? [1] https://dawn.cs.stanford.edu//benchmark/ImageNet/train.html https://dawn.cs.stanford.edu//benchmark/ImageNet/train.html
- sdenton4 8y agoNote the submission dates; the faster times have an extra six months.
- p1esk 8y agoHere are more recent results, showing that V100 is competitive with TPUv3: https://www.google.com/amp/s/cloudblog.withgoogle.com/products/ai-machine-learning/mlperf-benchmark-establishes-that-google-cloud-offers-the-most-accessible-scale-for-machine-learning-training https://www.google.com/amp/s/cloudblog.withgoogle.com/produc... I’m having a hard time imagining a task where TPU would be 10x faster than V100.