2 ms·
Every modern workflow orchestrator does those things you quote, and more. You make it sound like they're innovations when they're table stakes. Why wouldn't you
by esafak 2y ago
Every modern workflow orchestrator does those things you quote, and more. You make it sound like they're innovations when they're table stakes. Why wouldn't you just use Flyte or Kubeflow?
Also, the fact that they say that the alternative is "raw Spark" tells me either that they're confused, or not very good at explaining. Spark is used to execute tasks in a pipeline, not to orchestrate it.
- thenaturalist 2y agoWhile I generally tend to agree with your basic criticism, I think you need to keep in mind our perspectives might be biased due to limited data. Flyte went OSS what, 4 years ago? I'm not super familiar with it, but a) could have been that it was too unpolished at the time or b) requiring K8s to be a non-starter for some teams/ orgs. Same for Kubeflow. We also don't know for how long Koheesio existed within Nike. In short, there's a lot we don't know and there's a good chance the internal reasoning for investing in this made sense under certain circumstances in the past.
- esafak 2y agoThis project is two weeks old!
- thenaturalist 2y agoThe repo on GitHub is two weeks old? Doesn’t mean the project originated two weeks ago.