4 ms·

We already built what Amazon was looking for: An extensible deep-learning framework that works on distributed CPUs and GPUs heterogeneously, integrating with Sp

by vonnik 10y ago

We already built what Amazon was looking for: An extensible deep-learning framework that works on distributed CPUs and GPUs heterogeneously, integrating with Spark as an access layer to orchestrate multiple host threads. https://github.com/deeplearning4j https://github.com/deeplearning4j DL4J may be the DL library with the most sophisticated Spark integration at this point. The trick is to avoid using Spark as a computation layer, since it doesn't do that well. We're pushing a CuDNN wrapper tomorrow. http://deeplearning4j.org/quickstart http://deeplearning4j.org/quickstart Unlike DSSTNE, Tensorflow or CNTK, our deep learning library is neutral, and not designed with the intention of locking people into to a cloud service.

Fenntrek 10y ago
>Unlike DSSTNE, Tensorflow or CNTK, our deep learning library is neutral, and not designed with the intention of locking people into to a cloud service. How do any of these lock you into a cloud service?
- dgacmu 10y ago
  And to reinforce Fenntrek's point: One of the core reasons for open sourcing TensorFlow is to make sure people have the ability to take their code and run it somewhere else. The exact opposite of lock-in. (I can say that with some knowledge - I'm working there right now.) In fact, the cloud service for "hosted" TF isn't even out of alpha yet, but there are many people already using TF on their own hardware. I can't speak for MSFT or AMZN, but all three of these frameworks are completely open source with permissive licenses. I seriously doubt that any of these three providers has lock-in as a goal by releasing their stuff OSS. Goodwill? Absolutely. Getting people to learn the technology they care about internally? Certainly. Hoping that people will play with it and find that they want to run their models on 100 nodes rented from a cloud service? Seems likely. For MSFT, wanting to make sure there was a high-performance DNN framework for Windows? Would make sense to me. But lock-in by giving away your stuff in a way that lets anyone run it on their own hardware or a competitor's cloud? The DSSTNE benchmarks, for example, ran TensorFlow on AWS...
  - vonnik 10y ago
    I've spoken with the TF team and they've been clear that open-sourcing the lib was part of strategy to make Google Cloud more appealing. It was conceived as a lure, among other things.
    - th0ma5 10y ago
      Could be a great opportunity to push OpenCL development but I guess the whole world is still on CUDA :( so not quite open?
      - scottlegrand2 10y ago
        I'm gonna get in trouble saying this. But IMO OpenCL needs to die. IMO it's a half-baked open standard that's hard to use relative to CUDA. IMO what needs to happen is that AMD and Intel need to provide CUDA runtime (just say no to the driver API) API interfaces to their hardware. Evidence: OpenGL is a wonderful open standard. But OpenGL happened after nearly a decade of evolution of closed IrisGL that ironed out a lot of weird quirks that infest OpenCL to this day. I joined the CUDA team in March of 2006 (that's why I have 14 single-inventor CUDA patents, including one that is the basis of persistent RNNS by Baidu). IMO CUDA is a mature API. And CUDA needs to escape NVIDIA's vendor lock. AMD is attempting to make this happen with HIP and ROC. http://gpuopen.com/getting-started-with-boltzmann-components-platforms-installation/ http://gpuopen.com/getting-started-with-boltzmann-components... http://www.anandtech.com/show/9792/amd-sc15-boltzmann-initiative-announced-c-and-cuda-compilers-for-amd-gpus http://www.anandtech.com/show/9792/amd-sc15-boltzmann-initia... IMO as CUDA team member #6, we need to help AMD with this effort. To that end, we need to let go of OpenCL: I'm sorry but it's a dead-end. And we need to get as much of the existing CUDA code out there running on AMD hardware as we can. And I hope (but I don't think they'll agree because reasons(tm)) that Intel will provide a CUDA runtime API to Xeon Phi (Knights Landing or better) because CUDA is a better API for parallel computation than any other existing contender at this point. And that's my story and I'm sticking to it. But don't get me started on both Google and Apple refusing to expose OpenCL on mobile where it could have once made a real difference. That time has now passed. CUDA won, but we still don't have a CUDA or OpenCL interface on either Android or IOS.
scottlegrand2 10y ago
Wow, defensive much? Afraid of a little competition maybe? Anyway, DSSTNE itself is not in any way locked to any cloud service whatsoever. It is an Apache-licensed library with dependencies on a C++11 compiler, a 7.0 or later CUDA Toolkit and Kepler or better GPU, a C++11-friendly MPI library, netcdf, and libjsoncpp. That's it. Please stop insinuating otherwise. The article here shows how one could use DSSTNE with Spark for recommendations at scale. Speaking of which, how's your sparse data and model-parallel training support? Because as the author of DSSTNE, the poor support for these features in other frameworks was what forced to us to "roll our own" code in the first place. Everyone else was optimizing for ImageNet-winning CNNs (including NVIDIA). And there's nothing wrong with that, $25M+ companies have been built from that, but it just wasn't the use case here. Finally, cuSparse is cuSlow for datasets at Amazon. DSSTNE's hand-coded sparse kernels stomp on cuSparse in the same way that Neon's convolution kernels stomp on cuDNN. And there's nothing wrong with that either. CuSparse is a great choice for other sparse data problems, just not Amazon's (and I suspect many other companies) recommendations problems.
- deleted 10y ago
  [deleted]
AlexCoventry 10y ago
What's Deep Learning 4J's performance on the Soumith benchmarks? https://github.com/soumith/convnet-benchmarks https://github.com/soumith/convnet-benchmarks
- vonnik 10y ago
  We wrap Cudnn, which among the top performers.
  - scottlegrand2 10y ago
    Which doesn't answer the question he asked. CuDNN provides efficient convolution and RNN kernels. It is not a framework itself. There are plenty of ways to screw up aspects of a framework other than its convolution kernels.

by vonnik 10y ago

Fenntrek 10y ago

>Unlike DSSTNE, Tensorflow or CNTK, our deep learning library is neutral, and not designed with the intention of locking people into to a cloud service. How do any of these lock you into a cloud service?

dgacmu 10y ago
And to reinforce Fenntrek's point: One of the core reasons for open sourcing TensorFlow is to make sure people have the ability to take their code and run it somewhere else. The exact opposite of lock-in. (I can say that with some knowledge - I'm working there right now.) In fact, the cloud service for "hosted" TF isn't even out of alpha yet, but there are many people already using TF on their own hardware. I can't speak for MSFT or AMZN, but all three of these frameworks are completely open source with permissive licenses. I seriously doubt that any of these three providers has lock-in as a goal by releasing their stuff OSS. Goodwill? Absolutely. Getting people to learn the technology they care about internally? Certainly. Hoping that people will play with it and find that they want to run their models on 100 nodes rented from a cloud service? Seems likely. For MSFT, wanting to make sure there was a high-performance DNN framework for Windows? Would make sense to me. But lock-in by giving away your stuff in a way that lets anyone run it on their own hardware or a competitor's cloud? The DSSTNE benchmarks, for example, ran TensorFlow on AWS...
- vonnik 10y ago
  I've spoken with the TF team and they've been clear that open-sourcing the lib was part of strategy to make Google Cloud more appealing. It was conceived as a lure, among other things.
  - th0ma5 10y ago
    Could be a great opportunity to push OpenCL development but I guess the whole world is still on CUDA :( so not quite open?
    - scottlegrand2 10y ago
      I'm gonna get in trouble saying this. But IMO OpenCL needs to die. IMO it's a half-baked open standard that's hard to use relative to CUDA. IMO what needs to happen is that AMD and Intel need to provide CUDA runtime (just say no to the driver API) API interfaces to their hardware. Evidence: OpenGL is a wonderful open standard. But OpenGL happened after nearly a decade of evolution of closed IrisGL that ironed out a lot of weird quirks that infest OpenCL to this day. I joined the CUDA team in March of 2006 (that's why I have 14 single-inventor CUDA patents, including one that is the basis of persistent RNNS by Baidu). IMO CUDA is a mature API. And CUDA needs to escape NVIDIA's vendor lock. AMD is attempting to make this happen with HIP and ROC. http://gpuopen.com/getting-started-with-boltzmann-components-platforms-installation/ http://gpuopen.com/getting-started-with-boltzmann-components... http://www.anandtech.com/show/9792/amd-sc15-boltzmann-initiative-announced-c-and-cuda-compilers-for-amd-gpus http://www.anandtech.com/show/9792/amd-sc15-boltzmann-initia... IMO as CUDA team member #6, we need to help AMD with this effort. To that end, we need to let go of OpenCL: I'm sorry but it's a dead-end. And we need to get as much of the existing CUDA code out there running on AMD hardware as we can. And I hope (but I don't think they'll agree because reasons(tm)) that Intel will provide a CUDA runtime API to Xeon Phi (Knights Landing or better) because CUDA is a better API for parallel computation than any other existing contender at this point. And that's my story and I'm sticking to it. But don't get me started on both Google and Apple refusing to expose OpenCL on mobile where it could have once made a real difference. That time has now passed. CUDA won, but we still don't have a CUDA or OpenCL interface on either Android or IOS.

scottlegrand2 10y ago

Wow, defensive much? Afraid of a little competition maybe? Anyway, DSSTNE itself is not in any way locked to any cloud service whatsoever. It is an Apache-licensed library with dependencies on a C++11 compiler, a 7.0 or later CUDA Toolkit and Kepler or better GPU, a C++11-friendly MPI library, netcdf, and libjsoncpp. That's it. Please stop insinuating otherwise. The article here shows how one could use DSSTNE with Spark for recommendations at scale. Speaking of which, how's your sparse data and model-parallel training support? Because as the author of DSSTNE, the poor support for these features in other frameworks was what forced to us to "roll our own" code in the first place. Everyone else was optimizing for ImageNet-winning CNNs (including NVIDIA). And there's nothing wrong with that, $25M+ companies have been built from that, but it just wasn't the use case here. Finally, cuSparse is cuSlow for datasets at Amazon. DSSTNE's hand-coded sparse kernels stomp on cuSparse in the same way that Neon's convolution kernels stomp on cuDNN. And there's nothing wrong with that either. CuSparse is a great choice for other sparse data problems, just not Amazon's (and I suspect many other companies) recommendations problems.

deleted 10y ago
[deleted]

AlexCoventry 10y ago

What's Deep Learning 4J's performance on the Soumith benchmarks? https://github.com/soumith/convnet-benchmarks https://github.com/soumith/convnet-benchmarks

vonnik 10y ago
We wrap Cudnn, which among the top performers.
- scottlegrand2 10y ago
  Which doesn't answer the question he asked. CuDNN provides efficient convolution and RNN kernels. It is not a framework itself. There are plenty of ways to screw up aspects of a framework other than its convolution kernels.