4 ms·
I want to signal-boost this. TPU support on PyTorch is partial. You can run modeling computation on the TPU with PyTorch, but not the data-loading. And without
by arugulum 6y ago
I want to signal-boost this. TPU support on PyTorch is partial. You can run modeling computation on the TPU with PyTorch, but not the data-loading. And without the TPU's data-loading, you're significantly, significantly bottle-necked, to the point where you are often better off using GPUs. The reason why TensorFlow and TPUs synergize so well is that the TPUs themselves can consume data for training, allowing for massive scalability.
I have great respect for the PyTorch-TPU team, but I would recommend not heavily advertising PyTorch-TPU support until this major feature disparity is made up.
- wfalcon 6y agowe highlight these issues in our docs explicitly. https://pytorch-lightning.readthedocs.io/en/latest/tpu.html#tpu-support https://pytorch-lightning.readthedocs.io/en/latest/tpu.html#...
- marcinzm 6y agoCan you point out where exactly in those docs you highlight the issue? I just read the linked page and found no references to data loading limitations or performance limitations. Is it only in the video which isn't search indexed and few people would bother watching? edit: The page literally advertises the speed of TPUs with "In general, a single TPU is about as fast as 5 V100 GPUs!" which is the exact opposite of warning people.