7 ms·

Show HN: Eclipse Deeplearning4j

73 points by vonnik 9y ago

sandGorgon 9y ago
This is super, super cool! Love the work you guys have been putting out. How do you generally see the community mindshare in space especially with https://techcrunch.com/2017/02/13/yahoo-supercharges-tensorflow-with-apache-spark/ https://techcrunch.com/2017/02/13/yahoo-supercharges-tensorf... (you guys are also mentioned there) You guys run this at scale, but Yahoo's TensorFlowOnSpark does look very enticing with its "Easily migrate all existing TensorFlow programs with <10 lines of code change" punchline
- vonnik 9y ago
  Thanks! I'm not sure that Yahoo's TF on Spark is fully baked, but maybe its users can tell me more about that. I know it's not commercially supported. That's more a business model distinction, but it has technical consequences. Skymind built the DL4J stack from the ground up, so it's easy for us to maintain and extend, and we give enterprise a way to derisk adoption with a support contract. There is no equivalent offering for TensorFlow outside of GCE, let alone TF on Spark. In the end, it's more useful to think of DL4J as a complement to the Python ecosystem, rather than competing with it. We import TF models through Keras, and soon we'll import them directly to help people productionize the models they train with TF. That's important for large orgs committed to a JVM compute environment.
- agibsonccc 9y ago
  Doesn't matter. We don't need to be the #1 framework. We're going to be focused on model import and integrating with the big data ecosystem. None of these frameworks you're mentioning integrate properly, are really supported by an actual community, and don't get meaningful updates. The biggest problem you run in to pretty quickly is maintenance. TF and spark both update quickly. Every time someone has attempted to do a "TF on spark" they don't add ETL, proper JVM integrations (proper control of cuda and memory management from the JVM), there are also usually strange interactions with JVM/python run times. Also, please don't ignore the rest of dl4j. We have a whole suite of tools in there. It's not just a matrix lib with autodiff like TF and co is. TF in its own way is adding some of this stuff like TF records and some of their readers, but it's not going to add a lot in the way of things like connectors to kafka and a lot of the big data ecosystem. They've basically added HDFS..that was about it. Dl4j is also significantly easier to deploy. It's just a jar file or zip file..you don't need a blob of c++ just to run a model.
  - sandGorgon 9y ago
    So just trying to understand this - does dl4j become the essential middle layer if someone wants to run Tensorflow on spark seamlessly? Because it seemed on the surface that you guys replaced tf. If you guys offer a better ecosystem where tf and spark work together (but don't really rip out tf), then it is even more awesome.
    - agibsonccc 9y ago
      Not just TF: pytorch, keras,mxnet,.. Like I said: We won't be the #1 framework data scientists use. There's a ton of churn in that space yet. Granted, a ton of it is TF. Google is doing an amazing job now. We're trying to be a moderate middle ground. TF and pytorch and anything in python tends to be an interface. Core logic still operates in C/C++. That's great when you want speed, but you end up missing the benefits of the JVM (the tooling is great for monitoring and things folks need at scale). So we do everything via JNI where we push down the math one block with minimal overhead if any. That and if you want to write production code, you don't need to push logic down to c. You can do something JVM based instead, which gives you kotlin,scala,clojure,..
      - sandGorgon 9y ago
        i think this aspect of dl4j gets lost in the overall message. For me, this is much more powerful - "use dl4j if you want to have a seamless experience running tensorflow/keras/mxnet on spark". Because right now, it looks like it is tensorflow vs dl4j.
        agibsonccc 9y ago
        Honestly hard for me to care..They can both compete as well as integrate.No offense here but model import existing will become a common thing as things like ONNX become standard. We play up what's unique about Dl4j anyways. It's right in the name. We're square focused on the JVM. We also hedge our bets against spark. Most of our logic doesn't even run on spark. Spark is just a facilitator..it's not where anything that matters runs. We could just as easily use flink or apex here too. Those are also JVM based streaming engines. We can't overplay spark because most of our deployments won't even be with spark. We don't even use spark in our own production inference tools. We just deploy as a microservice.
deleted 9y ago
[deleted]
vonnik 9y ago
The interesting thing about this process is timing. You shouldn't really contribute a code base to a foundation until it's mature enough. I think any group of people developing an open-source project wants it to develop quickly and with healthy governance. Joining any foundation, whether it's Apache or Eclipse or Linux, sends a signal that they're mature, thinking about governance and want to make sure the product develops in a way that agrees with the community. But sometimes governance can get in the way of speed. What we found talking to Eclipse was that we could get the governance and keep the speed. Which means we'll be able to keep pushing DL4J forward without confusion. Fwiw, here's the DL4J site: https://deeplearning4j.org/ https://deeplearning4j.org/ Here are the repositories: https://github.com/deeplearning4j https://github.com/deeplearning4j And the Gitter community: https://gitter.im/deeplearning4j/deeplearning4j https://gitter.im/deeplearning4j/deeplearning4j
- mindcrime 9y ago
  DL4J strikes me as reasonably mature anyway, although you probably know more about that than I do. :-) Does this mean anything specific for Skymind as the commercial vendor behind DL4J?
  - vonnik 9y ago
    That's nice of you. :-) It's mature now, which is why this was the moment to move it into Eclipse. It means that DL4J & suite is now vendor neutral. On the Skymind side, we will continue to develop all those open-source projects, so you can expect a lot more cool stuff to come: interpretable models, better ETL, Keras as our Python API, vertical-specific apps for EDA, Robotics...
    - mindcrime 9y ago
      Very cool. I'm a DL4J fan. I actually did a talk at Tri-JUG a couple of weeks ago, on real-time machine learning and BPM, which featured DL4J as part of the tech stack. I'm also working on a SaaS offering around AI/ML and plan to include support for DL4J at some point. Hopefully at some point I can get to a place where I can make some useful contributions to the project.
crockpotveggies 9y ago
The amount of work we've done this year with Deeplearning4j on performance has been much higher than previous years. We brought DL4J up to par with community standards while maintaining the advantages of Java. I think what a lot of people don't realize is that a ton of effort has been made toward ETL and integration tooling. It's very difficult to train on multiple GPUs while maintaining performance of ETL. ETL is a scary hidden bottleneck. I'm very interested to see how Eclipse can continue to push development. I think the people who will especially benefit from this are devops/production teams operationalizing data science.
- stingraycharles 9y ago
  As someone who is mostly working in the field of data warehousing, ETL has a very meaning to me (Extract, Transform, Load). Is this the same as you’re talking about ?
  - crockpotveggies 9y ago
    Yes, so one of the core libraries within the DL4J project is datavec which is ETL-focused. One key problem that we discovered - and fixed - was that reading and transforming data for training could bottleneck a multi-GPU process. You spend a lot of $$$ on a deep learning computer, but making the library performant enough so that you could load data at the same rate the GPUs could consume it was challenging. This scales to about 4+ GPUs (depending on datatype) and we're building a datavec server so this can scale much larger. There are still good returns if you clean and transform your data and presave to disk, which helps with large machines such as a DGX. However, other bottlenecks still apply (which we are solving right now). I hope that answers your question. I consider the process of extracting records, transforming them, and loading them for training to be "ETL". I understand ETL also applies to other data consumption. *I should also note that if you wanted to use datavec for ETL and do not wish to train a deep learning model, it is quite useful for columnar data.
    - riku_iki 9y ago
      Why mix ETL and training? I am using TF, and in my workflow I first do all ETL in some separate process, dump all training/validation data into TFRecord file, and then my training program consumes it. Clear separation of concerns without any performance penalty. And I can iterate over training logic with various parameters as many times as I want without touching ETL.
stolsvik 9y ago
For anyone that hasn't checked out Deeplearning4j yet, and think that Python is the only place where the cool AI stuff happens, you really should check this project out! Their Gitter channel is absolutely kicking, where lots of the actual devs, including CTO and co-founder Adam Gibson, hang out and /really/ answer /any/ question - I find their responsiveness stunning! (I discussed and reported a minor improvement to the config builder, and it was implemented - literally - the very next day!) Just recently found the library myself, and I found it very refreshing to be able to utilize my decent Java competence in ML, instead of continuously having to hammer my way through limited Python experience to get things done. (Python also have limited multi-threading capabilities which I found annoying when wanting to do some larger ETL stuff "inline". Python also gives me a strong feeling of "this is a scripting language!": It is super fast and expressive when your needs are met by the vast set of (native) libraries - but when I just want to do something myself with raw code, things gets slower. I also love a properly typed language, which python obviously isn't!) Do notice that with dl4j, you still get full-on GPU acceleration via Nd4j, which is a "NumPy-style" multi-dim array system.
- agibsonccc 9y ago
  Thanks! Upcoming will be samediff which will allow a pytorch like api on the JVM as well as our python interface jumpy: https://github.com/deeplearning4j/jumpy https://github.com/deeplearning4j/jumpy The hope here is to be able to "meet" the python frameworks here and run side by side benchmarks that anyone who only knows TF could do. For those that don't know: DL4j is somewhere in between keras and TF flexibility wise. Samediff will be a bit "closer" to what people in python see/understand. The goal there is to import models from different frameworks and run them in production.
  - sandGorgon 9y ago
    I would like to chime in favor of python. The ecosystem is just too biased towards python to try and jump in anything else. Also looking at https://deeplearning4j.org/keras https://deeplearning4j.org/keras Could you have documentation which is helpful for the rest of us that use Linux ? Your choice of using docker is brilliant and hope that you make it significantly commandline driven and especially for GPU (nvidia-docker or something). Currently your documentation is extremly dependent on Kitematic and such.
agibsonccc 9y ago
For those wondering, why the heck are you guys java: Our tensor library has a python interface in the making: https://www.slideshare.net/agibsonccc/strata-beijing-2017-jumpy-a-python-interface-for-nd4j https://www.slideshare.net/agibsonccc/strata-beijing-2017-ju... Our goal with this will be then to write side by side benchmarks with the other libs that people can easily run. We know a lot of folks from python land won't jump over and we don't expect them to. The hope is that we're equivalent and can integrate better with a big data cluster. Dl4j is not trying to be TF. When I started dl4j 4 years ago, it was theano and torch as the primary frameworks. I wrote it for deployment in to production apps and for the hadoop/spark ecosystem. We will continue that going forward at the eclipse foundation as well as using dl4j in our product. Please check out the oreilly book as well (currently #2 on amazon right next to the goodfellow book): https://amazon.com/Deep-Learning-Practitioners-Josh-Patterson/dp/1491914254 https://amazon.com/Deep-Learning-Practitioners-Josh-Patterso... Email in profile if there's any specific questions.

73 points by vonnik 9y ago

This is super, super cool! Love the work you guys have been putting out. How do you generally see the community mindshare in space especially with https://techcrunch.com/2017/02/13/yahoo-supercharges-tensorflow-with-apache-spark/ https://techcrunch.com/2017/02/13/yahoo-supercharges-tensorf... (you guys are also mentioned there) You guys run this at scale, but Yahoo's TensorFlowOnSpark does look very enticing with its "Easily migrate all existing TensorFlow programs with <10 lines of code change" punchline

vonnik 9y ago
Thanks! I'm not sure that Yahoo's TF on Spark is fully baked, but maybe its users can tell me more about that. I know it's not commercially supported. That's more a business model distinction, but it has technical consequences. Skymind built the DL4J stack from the ground up, so it's easy for us to maintain and extend, and we give enterprise a way to derisk adoption with a support contract. There is no equivalent offering for TensorFlow outside of GCE, let alone TF on Spark. In the end, it's more useful to think of DL4J as a complement to the Python ecosystem, rather than competing with it. We import TF models through Keras, and soon we'll import them directly to help people productionize the models they train with TF. That's important for large orgs committed to a JVM compute environment.
agibsonccc 9y ago
Doesn't matter. We don't need to be the #1 framework. We're going to be focused on model import and integrating with the big data ecosystem. None of these frameworks you're mentioning integrate properly, are really supported by an actual community, and don't get meaningful updates. The biggest problem you run in to pretty quickly is maintenance. TF and spark both update quickly. Every time someone has attempted to do a "TF on spark" they don't add ETL, proper JVM integrations (proper control of cuda and memory management from the JVM), there are also usually strange interactions with JVM/python run times. Also, please don't ignore the rest of dl4j. We have a whole suite of tools in there. It's not just a matrix lib with autodiff like TF and co is. TF in its own way is adding some of this stuff like TF records and some of their readers, but it's not going to add a lot in the way of things like connectors to kafka and a lot of the big data ecosystem. They've basically added HDFS..that was about it. Dl4j is also significantly easier to deploy. It's just a jar file or zip file..you don't need a blob of c++ just to run a model.
- sandGorgon 9y ago
  So just trying to understand this - does dl4j become the essential middle layer if someone wants to run Tensorflow on spark seamlessly? Because it seemed on the surface that you guys replaced tf. If you guys offer a better ecosystem where tf and spark work together (but don't really rip out tf), then it is even more awesome.
  - agibsonccc 9y ago
    Not just TF: pytorch, keras,mxnet,.. Like I said: We won't be the #1 framework data scientists use. There's a ton of churn in that space yet. Granted, a ton of it is TF. Google is doing an amazing job now. We're trying to be a moderate middle ground. TF and pytorch and anything in python tends to be an interface. Core logic still operates in C/C++. That's great when you want speed, but you end up missing the benefits of the JVM (the tooling is great for monitoring and things folks need at scale). So we do everything via JNI where we push down the math one block with minimal overhead if any. That and if you want to write production code, you don't need to push logic down to c. You can do something JVM based instead, which gives you kotlin,scala,clojure,..
    - sandGorgon 9y ago
      i think this aspect of dl4j gets lost in the overall message. For me, this is much more powerful - "use dl4j if you want to have a seamless experience running tensorflow/keras/mxnet on spark". Because right now, it looks like it is tensorflow vs dl4j.
      - agibsonccc 9y ago
        Honestly hard for me to care..They can both compete as well as integrate.No offense here but model import existing will become a common thing as things like ONNX become standard. We play up what's unique about Dl4j anyways. It's right in the name. We're square focused on the JVM. We also hedge our bets against spark. Most of our logic doesn't even run on spark. Spark is just a facilitator..it's not where anything that matters runs. We could just as easily use flink or apex here too. Those are also JVM based streaming engines. We can't overplay spark because most of our deployments won't even be with spark. We don't even use spark in our own production inference tools. We just deploy as a microservice.

deleted 9y ago

[deleted]

vonnik 9y ago

The interesting thing about this process is timing. You shouldn't really contribute a code base to a foundation until it's mature enough. I think any group of people developing an open-source project wants it to develop quickly and with healthy governance. Joining any foundation, whether it's Apache or Eclipse or Linux, sends a signal that they're mature, thinking about governance and want to make sure the product develops in a way that agrees with the community. But sometimes governance can get in the way of speed. What we found talking to Eclipse was that we could get the governance and keep the speed. Which means we'll be able to keep pushing DL4J forward without confusion. Fwiw, here's the DL4J site: https://deeplearning4j.org/ https://deeplearning4j.org/ Here are the repositories: https://github.com/deeplearning4j https://github.com/deeplearning4j And the Gitter community: https://gitter.im/deeplearning4j/deeplearning4j https://gitter.im/deeplearning4j/deeplearning4j

mindcrime 9y ago
DL4J strikes me as reasonably mature anyway, although you probably know more about that than I do. :-) Does this mean anything specific for Skymind as the commercial vendor behind DL4J?
- vonnik 9y ago
  That's nice of you. :-) It's mature now, which is why this was the moment to move it into Eclipse. It means that DL4J & suite is now vendor neutral. On the Skymind side, we will continue to develop all those open-source projects, so you can expect a lot more cool stuff to come: interpretable models, better ETL, Keras as our Python API, vertical-specific apps for EDA, Robotics...
  - mindcrime 9y ago
    Very cool. I'm a DL4J fan. I actually did a talk at Tri-JUG a couple of weeks ago, on real-time machine learning and BPM, which featured DL4J as part of the tech stack. I'm also working on a SaaS offering around AI/ML and plan to include support for DL4J at some point. Hopefully at some point I can get to a place where I can make some useful contributions to the project.

crockpotveggies 9y ago

The amount of work we've done this year with Deeplearning4j on performance has been much higher than previous years. We brought DL4J up to par with community standards while maintaining the advantages of Java. I think what a lot of people don't realize is that a ton of effort has been made toward ETL and integration tooling. It's very difficult to train on multiple GPUs while maintaining performance of ETL. ETL is a scary hidden bottleneck. I'm very interested to see how Eclipse can continue to push development. I think the people who will especially benefit from this are devops/production teams operationalizing data science.

stingraycharles 9y ago
As someone who is mostly working in the field of data warehousing, ETL has a very meaning to me (Extract, Transform, Load). Is this the same as you’re talking about ?
- crockpotveggies 9y ago
  Yes, so one of the core libraries within the DL4J project is datavec which is ETL-focused. One key problem that we discovered - and fixed - was that reading and transforming data for training could bottleneck a multi-GPU process. You spend a lot of $$$ on a deep learning computer, but making the library performant enough so that you could load data at the same rate the GPUs could consume it was challenging. This scales to about 4+ GPUs (depending on datatype) and we're building a datavec server so this can scale much larger. There are still good returns if you clean and transform your data and presave to disk, which helps with large machines such as a DGX. However, other bottlenecks still apply (which we are solving right now). I hope that answers your question. I consider the process of extracting records, transforming them, and loading them for training to be "ETL". I understand ETL also applies to other data consumption. *I should also note that if you wanted to use datavec for ETL and do not wish to train a deep learning model, it is quite useful for columnar data.
  - riku_iki 9y ago
    Why mix ETL and training? I am using TF, and in my workflow I first do all ETL in some separate process, dump all training/validation data into TFRecord file, and then my training program consumes it. Clear separation of concerns without any performance penalty. And I can iterate over training logic with various parameters as many times as I want without touching ETL.

stolsvik 9y ago

For anyone that hasn't checked out Deeplearning4j yet, and think that Python is the only place where the cool AI stuff happens, you really should check this project out! Their Gitter channel is absolutely kicking, where lots of the actual devs, including CTO and co-founder Adam Gibson, hang out and /really/ answer /any/ question - I find their responsiveness stunning! (I discussed and reported a minor improvement to the config builder, and it was implemented - literally - the very next day!) Just recently found the library myself, and I found it very refreshing to be able to utilize my decent Java competence in ML, instead of continuously having to hammer my way through limited Python experience to get things done. (Python also have limited multi-threading capabilities which I found annoying when wanting to do some larger ETL stuff "inline". Python also gives me a strong feeling of "this is a scripting language!": It is super fast and expressive when your needs are met by the vast set of (native) libraries - but when I just want to do something myself with raw code, things gets slower. I also love a properly typed language, which python obviously isn't!) Do notice that with dl4j, you still get full-on GPU acceleration via Nd4j, which is a "NumPy-style" multi-dim array system.

agibsonccc 9y ago
Thanks! Upcoming will be samediff which will allow a pytorch like api on the JVM as well as our python interface jumpy: https://github.com/deeplearning4j/jumpy https://github.com/deeplearning4j/jumpy The hope here is to be able to "meet" the python frameworks here and run side by side benchmarks that anyone who only knows TF could do. For those that don't know: DL4j is somewhere in between keras and TF flexibility wise. Samediff will be a bit "closer" to what people in python see/understand. The goal there is to import models from different frameworks and run them in production.
- sandGorgon 9y ago
  I would like to chime in favor of python. The ecosystem is just too biased towards python to try and jump in anything else. Also looking at https://deeplearning4j.org/keras https://deeplearning4j.org/keras Could you have documentation which is helpful for the rest of us that use Linux ? Your choice of using docker is brilliant and hope that you make it significantly commandline driven and especially for GPU (nvidia-docker or something). Currently your documentation is extremly dependent on Kitematic and such.

agibsonccc 9y ago

For those wondering, why the heck are you guys java: Our tensor library has a python interface in the making: https://www.slideshare.net/agibsonccc/strata-beijing-2017-jumpy-a-python-interface-for-nd4j https://www.slideshare.net/agibsonccc/strata-beijing-2017-ju... Our goal with this will be then to write side by side benchmarks with the other libs that people can easily run. We know a lot of folks from python land won't jump over and we don't expect them to. The hope is that we're equivalent and can integrate better with a big data cluster. Dl4j is not trying to be TF. When I started dl4j 4 years ago, it was theano and torch as the primary frameworks. I wrote it for deployment in to production apps and for the hadoop/spark ecosystem. We will continue that going forward at the eclipse foundation as well as using dl4j in our product. Please check out the oreilly book as well (currently #2 on amazon right next to the goodfellow book): https://amazon.com/Deep-Learning-Practitioners-Josh-Patterson/dp/1491914254 https://amazon.com/Deep-Learning-Practitioners-Josh-Patterso... Email in profile if there's any specific questions.