3 ms·

Another potential source of friction I've wondered about regarding getting research workloads running on TPU2 is the heavy reliance of most of the popular DL fr

by maldeh 9y ago

Another potential source of friction I've wondered about regarding getting research workloads running on TPU2 is the heavy reliance of most of the popular DL frameworks on CUDA and CuDNN; I don't understand how one could port those libraries to a drastically different architecture such as this. Some DL frameworks have made overtures to OpenCL but it has not typically been a top priority [0]. Could this instead imply Google is working on its own library of low-level math primitives just for Tensorflow, and if so, how long would that take before it's competitive performance-wise? At any rate, having to support a different computing platform / API would be another blocker to general adoption. [0] https://github.com/tensorflow/tensorflow/issues/22 https://github.com/tensorflow/tensorflow/issues/22

deleted 9y ago
[deleted]
polishTar 9y ago
Would this not be solved by TensorFlow XLA? https://www.tensorflow.org/performance/xla/ https://www.tensorflow.org/performance/xla/
- maldeh 9y ago
  Per my understanding of XLA, it provides the right high-level abstraction for compiling the tensorflow computation graph under a range of architectures, CPU, NVIDIA GPU, mobile, etc. but it still delegates to a lower level domain-specific API like CUDA. This undertaking isn't something to be taken lightly; CuBLAS has some pretty cutting edge architecture-specific optimizations for batching operations for matrix multiplication that came from several years of research - and is arguably a massive competitive advantage of NVIDIA over AMD. Depending on the development state of such an API internally within Google, it could mean that the Cloud TPU isn't going to be ready for wide-spread commercial use for a good while, and is very much still in the research-and-development phase (which could explain why they're only opening it up for the research community right now).
  - maksimum 9y ago
    From what I gather from reading AMD's recent AMA is that the key is getting hardware and software guys [working together](https://lists.freedesktop.org/archives/dri-devel/2016-December/126808.html https://lists.freedesktop.org/archives/dri-devel/2016-Decemb...). For AMD the issue is that they can't afford to have hardware guys dedicated to helping optimize the software since they need to build the next and next next generation of hardware. On the other hand, Google surely has the pockets to get man hours and knowledge to (a) figure out what kind of software is needed (b) design from hardware level to meet their software needs. And if you caught any of their talks on [Tensorflow at Google IO](https://www.youtube.com/watch?v=5DknTFbcGVM https://www.youtube.com/watch?v=5DknTFbcGVM), it seems like their goal is to provide ultra-high level APIs like POST/GET, and high level python ones like Tensorflow pre-built and trained models. At that point Google can control the engineering, and as long as you run on their cloud platform you just have to worry about the high-level. I'm definitely not a fan of vendor lock-in, but it seems like a interesting product, and I'm curious to see what Google does in this space.
yaroslavvb 9y ago
You could specify computation in high level API like TensorFlow, and then have the framework pick the best implementations available (ie, CuDNN for GPU, MKL for CPU, something custom for TPU)
- maldeh 9y ago
  "something custom for TPU" is what seems interesting here. Researchers working on the TF research cloud probably wouldn't learn much about the sorts of workloads this architecture is suited to, if all they have access to is the high-level API. Would whatever iteration of the TPU API they have internally be something they'd release soon, at least to partners?

by maldeh 9y ago

deleted 9y ago

[deleted]

polishTar 9y ago

Would this not be solved by TensorFlow XLA? https://www.tensorflow.org/performance/xla/ https://www.tensorflow.org/performance/xla/

maldeh 9y ago
Per my understanding of XLA, it provides the right high-level abstraction for compiling the tensorflow computation graph under a range of architectures, CPU, NVIDIA GPU, mobile, etc. but it still delegates to a lower level domain-specific API like CUDA. This undertaking isn't something to be taken lightly; CuBLAS has some pretty cutting edge architecture-specific optimizations for batching operations for matrix multiplication that came from several years of research - and is arguably a massive competitive advantage of NVIDIA over AMD. Depending on the development state of such an API internally within Google, it could mean that the Cloud TPU isn't going to be ready for wide-spread commercial use for a good while, and is very much still in the research-and-development phase (which could explain why they're only opening it up for the research community right now).
- maksimum 9y ago
  From what I gather from reading AMD's recent AMA is that the key is getting hardware and software guys [working together](https://lists.freedesktop.org/archives/dri-devel/2016-December/126808.html https://lists.freedesktop.org/archives/dri-devel/2016-Decemb...). For AMD the issue is that they can't afford to have hardware guys dedicated to helping optimize the software since they need to build the next and next next generation of hardware. On the other hand, Google surely has the pockets to get man hours and knowledge to (a) figure out what kind of software is needed (b) design from hardware level to meet their software needs. And if you caught any of their talks on [Tensorflow at Google IO](https://www.youtube.com/watch?v=5DknTFbcGVM https://www.youtube.com/watch?v=5DknTFbcGVM), it seems like their goal is to provide ultra-high level APIs like POST/GET, and high level python ones like Tensorflow pre-built and trained models. At that point Google can control the engineering, and as long as you run on their cloud platform you just have to worry about the high-level. I'm definitely not a fan of vendor lock-in, but it seems like a interesting product, and I'm curious to see what Google does in this space.

yaroslavvb 9y ago

You could specify computation in high level API like TensorFlow, and then have the framework pick the best implementations available (ie, CuDNN for GPU, MKL for CPU, something custom for TPU)

maldeh 9y ago
"something custom for TPU" is what seems interesting here. Researchers working on the TF research cloud probably wouldn't learn much about the sorts of workloads this architecture is suited to, if all they have access to is the high-level API. Would whatever iteration of the TPU API they have internally be something they'd release soon, at least to partners?