5 ms·

> We're building a visual pipeline tool for data scientists. As a Sr. DS/ML Engineer, this doesn't speak to me.

by ellisv 6y ago

daemonk 6y ago
Yeah me too. I think visualizations are cool and can be useful in terms of monitoring the progress of analysis runs. But I can't see my self using a GUI for setting up pipelines when I can do it more explicitly through scripting or some kind of a command line based pipeline framework. However, maybe the use case here is for team collaboration where the pipelines are easier conveyed through visualizations.
- twunde 6y ago
  There's a large tradition of data analysis/BI tools using visual programming/ "no-code" tools, think Talend, etc. They're popular since data analysis typically contains a number of people without coding backgrounds. Even among data science teams, many don't have the engineering background of data engineers (and certainly there's a segment of data scientists who don't care as much about building the data pipelines and the cleaning that's necessary). Low code tools _can_ (but not necessarily will) effectively empower data teams.
- yannickperrenet 6y ago
  Thanks for your input. We are planning to also support an API and/or CLI (besides the GUI) to set up the pipelines through scripting. Depending on what we hear from the community we will definitely add this sooner rather than later. You make a great point for the use of monitoring the progress of a run. During pipeline runs the visual pipeline interface (also in view only mode during or after running pipeline runs as part of an experiment) can be helpful as a visual indication of where the pipeline failed, how long pipeline steps took to execute and whether certain pipeline steps are still running. Lastly, for team collaboration we think the visual interface is indeed helpful to explain (especially to the less technical people) the structure of the code, what exactly the pipeline sets out to solve and how the pipeline solves it. In addition, the possibility to add intermediary pipeline steps that serve as dashboards can be helpful for collaboration.
jpau 6y ago
I personally agree that this does not seem too useful for any DS team that needs to deploy a model to production. But there are a whole horde of DS teams whose outputs are basically PPT presentations. Think like pricing forecasts, decision modelling, marketing segmentation for product design, .... I think a common thread to these teams -- at least those that I've seen -- is that they consist of stats->DS backgrounds, and no eng->DS backgrounds. Many of these teams are orchestrating everything within the notebook. I've seen notebooks that contain complex workflows that extend to 10K LOC. I've lost sleep over such things.
- FridgeSeal 6y ago
  I agree, and so many data science and data engineering tools all seem to revolve around using notebooks, much to my frustration. I’ve worked in places whose data pipelines were built around seemingly infinite notebooks, all containing consistently poor software engineering. It’s been enough to make me vow to not let people write notebooks that go into prod under my watch lol. I’m constantly on the watch for software engineering focused tools that solve the issues, rather than data science/engineering focused tools. So many are inextricably linked into python as well, which doesn’t gel nicely with anywhere that has multiple languages in the codebase.
  - ricklamers 6y ago
    We'll allow your team to move parts of your code base from notebooks to scripts (.py/.sh/.R) to alleviate your frustration. That way you can keep using notebooks only for those parts where it makes most sense. They come together in pipelines that are JSON defined and git-versioned.
  - calebkaiser 6y ago
    Shameless plug, but I help maintain Cortex, and "software engineering focused tools that solve (ML) issues" is a neat summary of our entire philosophy. For example, instead of notebooks, our model serving platform (https://github.com/cortexlabs/cortex https://github.com/cortexlabs/cortex) uses YAML to structure deployments, and Python scripts to write inference APIs. It's still inextricably linked to Python, but only for writing your API. It's agnostic as to how the model itself is developed, so long as it can generate predictions.
ricklamers 6y ago
Maybe we need a better way to explain what we do. After reading through the comments and getting a better picture of what we're providing, does Orchest still not speak to you? Would love to learn more, as we're particularly keen on understanding how to communicate in the clearest way.
- JPKab 6y ago
  Disclaimer: I've recently at my company built and released a product that has a displayed DAG on screen and allows people to build ML pipelines in addition to the existing data prep and blend capability. My company is profitable and the ML product is doing well. We also have the ability to have jupyter notebooks execute in-line with the DAG. The comments on here are reflective of the fact that selling products like yours to typical coders is a dead-end. They don't like it or want it, not because it wouldn't be useful to them, but because they are attached to their current tooling and don't see a need for making their workflows more maintainable for non-coders. Some coders get it, after working in an office and realizing they are spending inordinate amounts of time tweaking their software for various data sources instead of focusing on the actual interesting stuff, but most don't. Your product has the challenge of falling in a valley between the domain experts who can't code and therefore won't have notebooks available to pull into each node, and the people who can code and have notebooks not wanting to spend money or use a dirty, filthy, evil GUI instead of their precious glue code. Your sweet spot of customers won't be on HN. They are the massive army of people with the job title "data scientist" who really aren't close to being actual data scientists. The enterprises are filled to the brim with them. Many of them can barely write Python or R, and are outright frauds. But plenty of them have a cursory knowledge of Python coding in Jupyter, and can actually do some Kaggle problems. However, basic ETL skills, web scraping, etc are foreign to them, and they have no ability to embed their shitty Jupyter notebook code into an ETL pipeline. These people aren't on HN. If they were, they wouldn't be so shitty at their job. Those people are your customers. The underqualified, borderline fraud data scientists who are EVERYWHERE. I sound like a cynical asshole. But there's a silver lining here: The best customers you can ever get aren't in that camp at all, and also would add a lot of value to their orgs using your tool. These folks are the domain expert analysts who really, really know their company's data and have the deep domain expertise. They are the people who data scientists have to talk to to be able to create value. You want to go after these people, and upskill them to basic python. People who know their business and it's data, and have just a little bit of data science skills create far more value than the opposite.

> We're building a visual pipeline tool for data scientists. As a Sr. DS/ML Engineer, this doesn't speak to me.

by ellisv 6y ago

> We're building a visual pipeline tool for data scientists. As a Sr. DS/ML Engineer, this doesn't speak to me.

Yeah me too. I think visualizations are cool and can be useful in terms of monitoring the progress of analysis runs. But I can't see my self using a GUI for setting up pipelines when I can do it more explicitly through scripting or some kind of a command line based pipeline framework. However, maybe the use case here is for team collaboration where the pipelines are easier conveyed through visualizations.

twunde 6y ago
There's a large tradition of data analysis/BI tools using visual programming/ "no-code" tools, think Talend, etc. They're popular since data analysis typically contains a number of people without coding backgrounds. Even among data science teams, many don't have the engineering background of data engineers (and certainly there's a segment of data scientists who don't care as much about building the data pipelines and the cleaning that's necessary). Low code tools _can_ (but not necessarily will) effectively empower data teams.
yannickperrenet 6y ago
Thanks for your input. We are planning to also support an API and/or CLI (besides the GUI) to set up the pipelines through scripting. Depending on what we hear from the community we will definitely add this sooner rather than later. You make a great point for the use of monitoring the progress of a run. During pipeline runs the visual pipeline interface (also in view only mode during or after running pipeline runs as part of an experiment) can be helpful as a visual indication of where the pipeline failed, how long pipeline steps took to execute and whether certain pipeline steps are still running. Lastly, for team collaboration we think the visual interface is indeed helpful to explain (especially to the less technical people) the structure of the code, what exactly the pipeline sets out to solve and how the pipeline solves it. In addition, the possibility to add intermediary pipeline steps that serve as dashboards can be helpful for collaboration.

jpau 6y ago

I personally agree that this does not seem too useful for any DS team that needs to deploy a model to production. But there are a whole horde of DS teams whose outputs are basically PPT presentations. Think like pricing forecasts, decision modelling, marketing segmentation for product design, .... I think a common thread to these teams -- at least those that I've seen -- is that they consist of stats->DS backgrounds, and no eng->DS backgrounds. Many of these teams are orchestrating everything within the notebook. I've seen notebooks that contain complex workflows that extend to 10K LOC. I've lost sleep over such things.

FridgeSeal 6y ago
I agree, and so many data science and data engineering tools all seem to revolve around using notebooks, much to my frustration. I’ve worked in places whose data pipelines were built around seemingly infinite notebooks, all containing consistently poor software engineering. It’s been enough to make me vow to not let people write notebooks that go into prod under my watch lol. I’m constantly on the watch for software engineering focused tools that solve the issues, rather than data science/engineering focused tools. So many are inextricably linked into python as well, which doesn’t gel nicely with anywhere that has multiple languages in the codebase.
- ricklamers 6y ago
  We'll allow your team to move parts of your code base from notebooks to scripts (.py/.sh/.R) to alleviate your frustration. That way you can keep using notebooks only for those parts where it makes most sense. They come together in pipelines that are JSON defined and git-versioned.
- calebkaiser 6y ago
  Shameless plug, but I help maintain Cortex, and "software engineering focused tools that solve (ML) issues" is a neat summary of our entire philosophy. For example, instead of notebooks, our model serving platform (https://github.com/cortexlabs/cortex https://github.com/cortexlabs/cortex) uses YAML to structure deployments, and Python scripts to write inference APIs. It's still inextricably linked to Python, but only for writing your API. It's agnostic as to how the model itself is developed, so long as it can generate predictions.

ricklamers 6y ago

Maybe we need a better way to explain what we do. After reading through the comments and getting a better picture of what we're providing, does Orchest still not speak to you? Would love to learn more, as we're particularly keen on understanding how to communicate in the clearest way.

JPKab 6y ago
Disclaimer: I've recently at my company built and released a product that has a displayed DAG on screen and allows people to build ML pipelines in addition to the existing data prep and blend capability. My company is profitable and the ML product is doing well. We also have the ability to have jupyter notebooks execute in-line with the DAG. The comments on here are reflective of the fact that selling products like yours to typical coders is a dead-end. They don't like it or want it, not because it wouldn't be useful to them, but because they are attached to their current tooling and don't see a need for making their workflows more maintainable for non-coders. Some coders get it, after working in an office and realizing they are spending inordinate amounts of time tweaking their software for various data sources instead of focusing on the actual interesting stuff, but most don't. Your product has the challenge of falling in a valley between the domain experts who can't code and therefore won't have notebooks available to pull into each node, and the people who can code and have notebooks not wanting to spend money or use a dirty, filthy, evil GUI instead of their precious glue code. Your sweet spot of customers won't be on HN. They are the massive army of people with the job title "data scientist" who really aren't close to being actual data scientists. The enterprises are filled to the brim with them. Many of them can barely write Python or R, and are outright frauds. But plenty of them have a cursory knowledge of Python coding in Jupyter, and can actually do some Kaggle problems. However, basic ETL skills, web scraping, etc are foreign to them, and they have no ability to embed their shitty Jupyter notebook code into an ETL pipeline. These people aren't on HN. If they were, they wouldn't be so shitty at their job. Those people are your customers. The underqualified, borderline fraud data scientists who are EVERYWHERE. I sound like a cynical asshole. But there's a silver lining here: The best customers you can ever get aren't in that camp at all, and also would add a lot of value to their orgs using your tool. These folks are the domain expert analysts who really, really know their company's data and have the deep domain expertise. They are the people who data scientists have to talk to to be able to create value. You want to go after these people, and upskill them to basic python. People who know their business and it's data, and have just a little bit of data science skills create far more value than the opposite.