4 ms·

TensorFlow has kind of won the Python deep-learning community, although frameworks like Keras[0] make it easier to use. While Python programmers make up the maj

by vonnik 10y ago

TensorFlow has kind of won the Python deep-learning community, although frameworks like Keras[0] make it easier to use. While Python programmers make up the majority of deep-learning practitioners, they don't have much penetration in enterprise, which is chiefly JVM and lower-level languages. Deeplearning4j [1] has won deep learning on the JVM. Nothing else comes close. (Disclosure: I'm one of its creators.) DL4J has the most sophisticated Spark integration, plugs into Kafka and works as a Hadoop job. It does all that while running on distributed GPUs, multiple CPUs and heterogenous hardware. It wraps cuDNN, which is faster than Tensorflow. Finally, part of our stack, JavaCPP [2], is used by the TF communities as well as other big company projects to bridge the gap between Java and C++. Dev mindshare in the form of Github stars doesn't equal revenue. Finally, there are a number of strategic problems companies face when adopting a Google project. First, they are leery of depending on Google when they also compete with it. This is particularly true because Google doesn't open source a lot of its best work, like code from DeepMind. So Tensorflow adopters are getting the leftovers in a way, and they're kneecapped in the AI race. Secondly, Tensorflow is not platform neutral. It's optimized for TPUs, which are not publicly available. It's a play to attract people to the Google Cloud. [0] http://keras.io/ http://keras.io/ [1] http://deeplearning4j.org/ http://deeplearning4j.org/ [2] https://github.com/bytedeco/javacpp https://github.com/bytedeco/javacpp

cs702 10y ago
Upvoting, because I agree that JVM and .NET shops, like commercial banks and industrial manufacturers, are more likely to use Deeplearning4j (DL4J) and CNTK, respectively, than TensorFlow (or anything else, really). However, those corporate shops are typically many years behind the state-of-the-art in AI, and in many, if not most, cases, they will likely end up paying for higher-level "AI services" provided by a third party than building new kinds of applications in-house with lower-level frameworks. My view is that those new AI services will be created by startups we haven't heard about yet, led by developers outside of Corporate America who by and large are betting on TensorFlow and non-Oracle, non-Microsoft stacks. Google's competitors will of course want to use their own in-house frameworks regardless. -- PS. By the way, I think DL4J is awesome. Great job!
- agibsonccc 10y ago
  (Disclosure: Vonnik's cofounder creator of dl4j) That's more than likely the valley echo chamber. These kinds of startups are going to continue to appeal to people like themselves getting acquired by google eventually. The same scientists who create these services have zero clue about the sales process in enterprise and can't often figure out how to sell in to these corporations and actually create some kind of long term value. Our differentiation in this space is being able to do EXACTLY that while also adding features that are actually useful. By the time they have, they're usually bored and realize they just want to work on cool problems and therefore: sell. Most research out there generated quickly is actually useless for products. I'd rather implement every 5th iteration of a technique that has significantly better results than jump on the feature treadmill.
  - cs702 10y ago
    As much as I want you guys to succeed, I must disagree with this comment, because there are an awful LOT of people in the valley who know how to build scalable enterprise-sales organizations, and "AI service" startups with traction will have little trouble finding and recruiting those people. I would not bet against the valley's "how to build a giant business" expertise.
    - agibsonccc 10y ago
      We're far from that. Much of what we actually sell are solutions focused on anomaly detection. We actually do most of our business in asia where the on prem market is much larger and people move faster. What I'm trying to point out here is that the people who run these startups are typically Phds with little interest in building an actual business. Many of these deep learning startups are built to be quick flips while having fun for a few years building something cool. That in and of itself is fine. They build something differentiated and valuable, get paid a handsome chunk of money. No one really loses when that happens, it's just different when you want to go to market. That being said, you have every right to bet against us. That's how many large companies start is with detractors. 90% of startups also fail. We have a different opinion on how it's done. Only time will tell whether we succeed or not. That being said: Where's your deep learning startup? ;) At least we're trying something different.
ajtulloch 10y ago
"It wraps cuDNN, which is faster than Tensorflow." TF uses CuDNN, so this is a nonsensical statement.
- vonnik 10y ago
  No, it's not. Why does TensorFlow have separate benchmarks for Soumith's convnet comparison that show it to be significantly slower? https://github.com/soumith/convnet-benchmarks https://github.com/soumith/convnet-benchmarks
  - p1esk 10y ago
    It says right there: "Some of the frameworks like TensorFlow and Chainer are benchmarked with CuDNN, but it is not explicitly mentioned" So yeah, I'm also not sure what you meant by your statement.
    - vonnik 10y ago
      But cuDNN itself is faster than those frameworks, so they are doing something to slow themselves down, and my statement stands.
      - p1esk 10y ago
        Again, this does not make sense. CuDNN is the highly optimized code to perform a specific numerical calculation (e.g. convolution) on Nvidia GPUs. Frameworks such as TensorFlow or Deeplearning4j can use CuDNN to speed up its convnet calculations, but they don't have to. In summary, your statement is like saying: "Numpy is faster than Python". It feels strange having to explain these things to a creator of a NN framework.
        vonnik 10y ago
        That's a false analogy. What I'm saying is not like "Numpy is faster than Python"; it's like "Numpy is faster than some scientific computing that the creators of a Python framework hand-rolled separately." Some frameworks write their own CUDA kernels for operations beyond what cuDNN offers, and also wrap cuDNN. So certain operations may run faster or slower depending on how you choose to execute them with the framework.

by vonnik 10y ago

cs702 10y ago

Upvoting, because I agree that JVM and .NET shops, like commercial banks and industrial manufacturers, are more likely to use Deeplearning4j (DL4J) and CNTK, respectively, than TensorFlow (or anything else, really). However, those corporate shops are typically many years behind the state-of-the-art in AI, and in many, if not most, cases, they will likely end up paying for higher-level "AI services" provided by a third party than building new kinds of applications in-house with lower-level frameworks. My view is that those new AI services will be created by startups we haven't heard about yet, led by developers outside of Corporate America who by and large are betting on TensorFlow and non-Oracle, non-Microsoft stacks. Google's competitors will of course want to use their own in-house frameworks regardless. -- PS. By the way, I think DL4J is awesome. Great job!

agibsonccc 10y ago
(Disclosure: Vonnik's cofounder creator of dl4j) That's more than likely the valley echo chamber. These kinds of startups are going to continue to appeal to people like themselves getting acquired by google eventually. The same scientists who create these services have zero clue about the sales process in enterprise and can't often figure out how to sell in to these corporations and actually create some kind of long term value. Our differentiation in this space is being able to do EXACTLY that while also adding features that are actually useful. By the time they have, they're usually bored and realize they just want to work on cool problems and therefore: sell. Most research out there generated quickly is actually useless for products. I'd rather implement every 5th iteration of a technique that has significantly better results than jump on the feature treadmill.
- cs702 10y ago
  As much as I want you guys to succeed, I must disagree with this comment, because there are an awful LOT of people in the valley who know how to build scalable enterprise-sales organizations, and "AI service" startups with traction will have little trouble finding and recruiting those people. I would not bet against the valley's "how to build a giant business" expertise.
  - agibsonccc 10y ago
    We're far from that. Much of what we actually sell are solutions focused on anomaly detection. We actually do most of our business in asia where the on prem market is much larger and people move faster. What I'm trying to point out here is that the people who run these startups are typically Phds with little interest in building an actual business. Many of these deep learning startups are built to be quick flips while having fun for a few years building something cool. That in and of itself is fine. They build something differentiated and valuable, get paid a handsome chunk of money. No one really loses when that happens, it's just different when you want to go to market. That being said, you have every right to bet against us. That's how many large companies start is with detractors. 90% of startups also fail. We have a different opinion on how it's done. Only time will tell whether we succeed or not. That being said: Where's your deep learning startup? ;) At least we're trying something different.

ajtulloch 10y ago

"It wraps cuDNN, which is faster than Tensorflow." TF uses CuDNN, so this is a nonsensical statement.

vonnik 10y ago
No, it's not. Why does TensorFlow have separate benchmarks for Soumith's convnet comparison that show it to be significantly slower? https://github.com/soumith/convnet-benchmarks https://github.com/soumith/convnet-benchmarks
- p1esk 10y ago
  It says right there: "Some of the frameworks like TensorFlow and Chainer are benchmarked with CuDNN, but it is not explicitly mentioned" So yeah, I'm also not sure what you meant by your statement.
  - vonnik 10y ago
    But cuDNN itself is faster than those frameworks, so they are doing something to slow themselves down, and my statement stands.
    - p1esk 10y ago
      Again, this does not make sense. CuDNN is the highly optimized code to perform a specific numerical calculation (e.g. convolution) on Nvidia GPUs. Frameworks such as TensorFlow or Deeplearning4j can use CuDNN to speed up its convnet calculations, but they don't have to. In summary, your statement is like saying: "Numpy is faster than Python". It feels strange having to explain these things to a creator of a NN framework.
      - vonnik 10y ago
        That's a false analogy. What I'm saying is not like "Numpy is faster than Python"; it's like "Numpy is faster than some scientific computing that the creators of a Python framework hand-rolled separately." Some frameworks write their own CUDA kernels for operations beyond what cuDNN offers, and also wrap cuDNN. So certain operations may run faster or slower depending on how you choose to execute them with the framework.