4 ms·
Integrations and deployment. We are targeting data engineers with this not data scientists. A lot of our user base are people who want to take what the python
by agibsonccc 9y ago
Integrations and deployment. We are targeting data engineers with this not data scientists.
A lot of our user base are people who want to take what the python folks did for creating a feature vector from raw data -> import a model (note this is all in java) and run the training pipeline and test pipeline in the same place.
The goal is to provide an opinionated set of tools for doing this.
Upside here:
It's obvious how to both pre process data and create ndarrays from them. Tighter integration also allows us to make some optimizations under the hood with how memory allocation is handled as well as allows us to target different data types like images + sound as well as databases in the same place.
What I'm guessing here is: You're a data scientist focused on building the models. Someone has to take that code and put it in production. If you work at a startup, production might be "TF serving". If you're a fortune 500,
you're likely not deploying that. People we've worked with are usually constrained in some way (especially by the JVM) .
You aren't our target audience, it's your colleagues.
We'll work backwards from that by adding our python interface and the like, but largely we want to solve a cross team concern after models are built.