4 ms·

The amount of work we've done this year with Deeplearning4j on performance has been much higher than previous years. We brought DL4J up to par with community st

The amount of work we've done this year with Deeplearning4j on performance has been much higher than previous years. We brought DL4J up to par with community standards while maintaining the advantages of Java. I think what a lot of people don't realize is that a ton of effort has been made toward ETL and integration tooling. It's very difficult to train on multiple GPUs while maintaining performance of ETL. ETL is a scary hidden bottleneck. I'm very interested to see how Eclipse can continue to push development. I think the people who will especially benefit from this are devops/production teams operationalizing data science.

stingraycharles 9y ago
As someone who is mostly working in the field of data warehousing, ETL has a very meaning to me (Extract, Transform, Load). Is this the same as you’re talking about ?
- crockpotveggies 9y ago
  Yes, so one of the core libraries within the DL4J project is datavec which is ETL-focused. One key problem that we discovered - and fixed - was that reading and transforming data for training could bottleneck a multi-GPU process. You spend a lot of $$$ on a deep learning computer, but making the library performant enough so that you could load data at the same rate the GPUs could consume it was challenging. This scales to about 4+ GPUs (depending on datatype) and we're building a datavec server so this can scale much larger. There are still good returns if you clean and transform your data and presave to disk, which helps with large machines such as a DGX. However, other bottlenecks still apply (which we are solving right now). I hope that answers your question. I consider the process of extracting records, transforming them, and loading them for training to be "ETL". I understand ETL also applies to other data consumption. *I should also note that if you wanted to use datavec for ETL and do not wish to train a deep learning model, it is quite useful for columnar data.
  - riku_iki 9y ago
    Why mix ETL and training? I am using TF, and in my workflow I first do all ETL in some separate process, dump all training/validation data into TFRecord file, and then my training program consumes it. Clear separation of concerns without any performance penalty. And I can iterate over training logic with various parameters as many times as I want without touching ETL.
    - agibsonccc 9y ago
      Integrations and deployment. We are targeting data engineers with this not data scientists. A lot of our user base are people who want to take what the python folks did for creating a feature vector from raw data -> import a model (note this is all in java) and run the training pipeline and test pipeline in the same place. The goal is to provide an opinionated set of tools for doing this. Upside here: It's obvious how to both pre process data and create ndarrays from them. Tighter integration also allows us to make some optimizations under the hood with how memory allocation is handled as well as allows us to target different data types like images + sound as well as databases in the same place. What I'm guessing here is: You're a data scientist focused on building the models. Someone has to take that code and put it in production. If you work at a startup, production might be "TF serving". If you're a fortune 500, you're likely not deploying that. People we've worked with are usually constrained in some way (especially by the JVM) . You aren't our target audience, it's your colleagues. We'll work backwards from that by adding our python interface and the like, but largely we want to solve a cross team concern after models are built.
    - stolsvik 9y ago
      Because you might want, as I definitely wanted, to also iterate over the feature engineering, not only the network and training parameters. Thus, doing ETL "inline" is really cool, and speeds up your iteration. It is also easy once you have a proper language that supports multi-threading. Since you do it "full speed" with full utilization of the GPU, there is no disadvantage of doing the ETL while training.
      - deleted 9y ago
- vonnik 9y ago
  Fwiw, here are the links for our ETL tool DataVec (it vectorizes data, or tensorizes it if you prefer): https://github.com/deeplearning4j/datavec https://github.com/deeplearning4j/datavec https://deeplearning4j.org/datavec https://deeplearning4j.org/datavec The thing to remember is that this is ETL focused on machine learning. It's not any old set of transforms. It's transforms that help us normalize, standardize and finally tensorize various data types, be they images, video, text or time series.

stingraycharles 9y ago

As someone who is mostly working in the field of data warehousing, ETL has a very meaning to me (Extract, Transform, Load). Is this the same as you’re talking about ?

crockpotveggies 9y ago
Yes, so one of the core libraries within the DL4J project is datavec which is ETL-focused. One key problem that we discovered - and fixed - was that reading and transforming data for training could bottleneck a multi-GPU process. You spend a lot of $$$ on a deep learning computer, but making the library performant enough so that you could load data at the same rate the GPUs could consume it was challenging. This scales to about 4+ GPUs (depending on datatype) and we're building a datavec server so this can scale much larger. There are still good returns if you clean and transform your data and presave to disk, which helps with large machines such as a DGX. However, other bottlenecks still apply (which we are solving right now). I hope that answers your question. I consider the process of extracting records, transforming them, and loading them for training to be "ETL". I understand ETL also applies to other data consumption. *I should also note that if you wanted to use datavec for ETL and do not wish to train a deep learning model, it is quite useful for columnar data.
- riku_iki 9y ago
  Why mix ETL and training? I am using TF, and in my workflow I first do all ETL in some separate process, dump all training/validation data into TFRecord file, and then my training program consumes it. Clear separation of concerns without any performance penalty. And I can iterate over training logic with various parameters as many times as I want without touching ETL.
  - agibsonccc 9y ago
    Integrations and deployment. We are targeting data engineers with this not data scientists. A lot of our user base are people who want to take what the python folks did for creating a feature vector from raw data -> import a model (note this is all in java) and run the training pipeline and test pipeline in the same place. The goal is to provide an opinionated set of tools for doing this. Upside here: It's obvious how to both pre process data and create ndarrays from them. Tighter integration also allows us to make some optimizations under the hood with how memory allocation is handled as well as allows us to target different data types like images + sound as well as databases in the same place. What I'm guessing here is: You're a data scientist focused on building the models. Someone has to take that code and put it in production. If you work at a startup, production might be "TF serving". If you're a fortune 500, you're likely not deploying that. People we've worked with are usually constrained in some way (especially by the JVM) . You aren't our target audience, it's your colleagues. We'll work backwards from that by adding our python interface and the like, but largely we want to solve a cross team concern after models are built.
  - stolsvik 9y ago
    Because you might want, as I definitely wanted, to also iterate over the feature engineering, not only the network and training parameters. Thus, doing ETL "inline" is really cool, and speeds up your iteration. It is also easy once you have a proper language that supports multi-threading. Since you do it "full speed" with full utilization of the GPU, there is no disadvantage of doing the ETL while training.
    - deleted 9y ago
vonnik 9y ago
Fwiw, here are the links for our ETL tool DataVec (it vectorizes data, or tensorizes it if you prefer): https://github.com/deeplearning4j/datavec https://github.com/deeplearning4j/datavec https://deeplearning4j.org/datavec https://deeplearning4j.org/datavec The thing to remember is that this is ETL focused on machine learning. It's not any old set of transforms. It's transforms that help us normalize, standardize and finally tensorize various data types, be they images, video, text or time series.