3 ms·
I was recently surprised to discover that Jax can't use a TPU's CPU, and that there are no plans to add this to Jax. https://github.com/google/jax/issues/2108#i
by sillysaurusx 7y ago
I was recently surprised to discover that Jax can't use a TPU's CPU, and that there are no plans to add this to Jax. https://github.com/google/jax/issues/2108#issuecomment-581541862 https://github.com/google/jax/issues/2108#issuecomment-58154...
A TPU's CPU is the only reason that TPUs are able to get such high performance on MLPerf benchmarks like imagenet resnet training. https://mlperf.org/training-results-0-6 https://mlperf.org/training-results-0-6
They do infeed processing (image transforms, etc) on the TPU's CPU. Then the results are fed to each TPU core.
Without this capability, I don't know how you'd feed the TPUs with data in a timely fashion. It seems like your input will be starved.
Hopefully they'll bring jax to parity with tensorflow in this regard soon. Otherwise, given that jax is a serious tensorflow competitor, I'm not sure how the future of TPUs will play out.
(If it sounds like this is just a minor feature, consider how it would sound to say "We're selling this car, and it can go fast, but it has no seats." Kind of a crucial feature of a car.)
Still, I think this is just a passing issue. There's no way that Google is going to let their TPU fleet languish. Not when they bring in >$1M/yr per TPU pod commitment.
- yablak 7y agoYou can use tensorflow's tf.data, I think.
- sillysaurusx 7y agoSort of. The way you do it is to scope operations to tf.device(None), which selects the TPU's CPU. (I think it's equivalent to using the first device in sess.list_devices(), which is the CPU.) You can scope operations in tf.data, sure, but you can also execute arbitrary training operations. I use this technique to train GPT-2 117M with a 25k context window, which requires about 300GB memory. Only a TPU's CPU has so much. That's why it was surprising to hear Jax can't do that. It's one of the best features of a TPU.