5 ms·

This is super, super cool! Love the work you guys have been putting out. How do you generally see the community mindshare in space especially with https://tech

This is super, super cool! Love the work you guys have been putting out. How do you generally see the community mindshare in space especially with https://techcrunch.com/2017/02/13/yahoo-supercharges-tensorflow-with-apache-spark/ https://techcrunch.com/2017/02/13/yahoo-supercharges-tensorf... (you guys are also mentioned there) You guys run this at scale, but Yahoo's TensorFlowOnSpark does look very enticing with its "Easily migrate all existing TensorFlow programs with <10 lines of code change" punchline

vonnik 9y ago
Thanks! I'm not sure that Yahoo's TF on Spark is fully baked, but maybe its users can tell me more about that. I know it's not commercially supported. That's more a business model distinction, but it has technical consequences. Skymind built the DL4J stack from the ground up, so it's easy for us to maintain and extend, and we give enterprise a way to derisk adoption with a support contract. There is no equivalent offering for TensorFlow outside of GCE, let alone TF on Spark. In the end, it's more useful to think of DL4J as a complement to the Python ecosystem, rather than competing with it. We import TF models through Keras, and soon we'll import them directly to help people productionize the models they train with TF. That's important for large orgs committed to a JVM compute environment.
agibsonccc 9y ago
Doesn't matter. We don't need to be the #1 framework. We're going to be focused on model import and integrating with the big data ecosystem. None of these frameworks you're mentioning integrate properly, are really supported by an actual community, and don't get meaningful updates. The biggest problem you run in to pretty quickly is maintenance. TF and spark both update quickly. Every time someone has attempted to do a "TF on spark" they don't add ETL, proper JVM integrations (proper control of cuda and memory management from the JVM), there are also usually strange interactions with JVM/python run times. Also, please don't ignore the rest of dl4j. We have a whole suite of tools in there. It's not just a matrix lib with autodiff like TF and co is. TF in its own way is adding some of this stuff like TF records and some of their readers, but it's not going to add a lot in the way of things like connectors to kafka and a lot of the big data ecosystem. They've basically added HDFS..that was about it. Dl4j is also significantly easier to deploy. It's just a jar file or zip file..you don't need a blob of c++ just to run a model.
- sandGorgon 9y ago
  So just trying to understand this - does dl4j become the essential middle layer if someone wants to run Tensorflow on spark seamlessly? Because it seemed on the surface that you guys replaced tf. If you guys offer a better ecosystem where tf and spark work together (but don't really rip out tf), then it is even more awesome.
  - agibsonccc 9y ago
    Not just TF: pytorch, keras,mxnet,.. Like I said: We won't be the #1 framework data scientists use. There's a ton of churn in that space yet. Granted, a ton of it is TF. Google is doing an amazing job now. We're trying to be a moderate middle ground. TF and pytorch and anything in python tends to be an interface. Core logic still operates in C/C++. That's great when you want speed, but you end up missing the benefits of the JVM (the tooling is great for monitoring and things folks need at scale). So we do everything via JNI where we push down the math one block with minimal overhead if any. That and if you want to write production code, you don't need to push logic down to c. You can do something JVM based instead, which gives you kotlin,scala,clojure,..
    - sandGorgon 9y ago
      i think this aspect of dl4j gets lost in the overall message. For me, this is much more powerful - "use dl4j if you want to have a seamless experience running tensorflow/keras/mxnet on spark". Because right now, it looks like it is tensorflow vs dl4j.
      - agibsonccc 9y ago
        Honestly hard for me to care..They can both compete as well as integrate.No offense here but model import existing will become a common thing as things like ONNX become standard. We play up what's unique about Dl4j anyways. It's right in the name. We're square focused on the JVM. We also hedge our bets against spark. Most of our logic doesn't even run on spark. Spark is just a facilitator..it's not where anything that matters runs. We could just as easily use flink or apex here too. Those are also JVM based streaming engines. We can't overplay spark because most of our deployments won't even be with spark. We don't even use spark in our own production inference tools. We just deploy as a microservice.

vonnik 9y ago

Thanks! I'm not sure that Yahoo's TF on Spark is fully baked, but maybe its users can tell me more about that. I know it's not commercially supported. That's more a business model distinction, but it has technical consequences. Skymind built the DL4J stack from the ground up, so it's easy for us to maintain and extend, and we give enterprise a way to derisk adoption with a support contract. There is no equivalent offering for TensorFlow outside of GCE, let alone TF on Spark. In the end, it's more useful to think of DL4J as a complement to the Python ecosystem, rather than competing with it. We import TF models through Keras, and soon we'll import them directly to help people productionize the models they train with TF. That's important for large orgs committed to a JVM compute environment.

agibsonccc 9y ago

Doesn't matter. We don't need to be the #1 framework. We're going to be focused on model import and integrating with the big data ecosystem. None of these frameworks you're mentioning integrate properly, are really supported by an actual community, and don't get meaningful updates. The biggest problem you run in to pretty quickly is maintenance. TF and spark both update quickly. Every time someone has attempted to do a "TF on spark" they don't add ETL, proper JVM integrations (proper control of cuda and memory management from the JVM), there are also usually strange interactions with JVM/python run times. Also, please don't ignore the rest of dl4j. We have a whole suite of tools in there. It's not just a matrix lib with autodiff like TF and co is. TF in its own way is adding some of this stuff like TF records and some of their readers, but it's not going to add a lot in the way of things like connectors to kafka and a lot of the big data ecosystem. They've basically added HDFS..that was about it. Dl4j is also significantly easier to deploy. It's just a jar file or zip file..you don't need a blob of c++ just to run a model.

sandGorgon 9y ago
So just trying to understand this - does dl4j become the essential middle layer if someone wants to run Tensorflow on spark seamlessly? Because it seemed on the surface that you guys replaced tf. If you guys offer a better ecosystem where tf and spark work together (but don't really rip out tf), then it is even more awesome.
- agibsonccc 9y ago
  Not just TF: pytorch, keras,mxnet,.. Like I said: We won't be the #1 framework data scientists use. There's a ton of churn in that space yet. Granted, a ton of it is TF. Google is doing an amazing job now. We're trying to be a moderate middle ground. TF and pytorch and anything in python tends to be an interface. Core logic still operates in C/C++. That's great when you want speed, but you end up missing the benefits of the JVM (the tooling is great for monitoring and things folks need at scale). So we do everything via JNI where we push down the math one block with minimal overhead if any. That and if you want to write production code, you don't need to push logic down to c. You can do something JVM based instead, which gives you kotlin,scala,clojure,..
  - sandGorgon 9y ago
    i think this aspect of dl4j gets lost in the overall message. For me, this is much more powerful - "use dl4j if you want to have a seamless experience running tensorflow/keras/mxnet on spark". Because right now, it looks like it is tensorflow vs dl4j.
    - agibsonccc 9y ago
      Honestly hard for me to care..They can both compete as well as integrate.No offense here but model import existing will become a common thing as things like ONNX become standard. We play up what's unique about Dl4j anyways. It's right in the name. We're square focused on the JVM. We also hedge our bets against spark. Most of our logic doesn't even run on spark. Spark is just a facilitator..it's not where anything that matters runs. We could just as easily use flink or apex here too. Those are also JVM based streaming engines. We can't overplay spark because most of our deployments won't even be with spark. We don't even use spark in our own production inference tools. We just deploy as a microservice.