5 ms·

"Workflow" is such a loaded/broad/bifurcated term. In this case we seem to be talking about "Scientific Computing Workflow" or some sort of data pipeline. (As

by fowl2 7y ago

"Workflow" is such a loaded/broad/bifurcated term. In this case we seem to be talking about "Scientific Computing Workflow" or some sort of data pipeline. (As opposed to some sort of "Enterprise" workflow)

oeuviz 7y ago
Fully agree. It seems nobody understands it in the sense of Business Processes/Workflow modelling anymore and it is arbitrary used for "something to do".
- liotier 7y ago
  I did expect business process modeling... I was confused !
lewisjoe 7y ago
Yes, "Workflow" has to be one of the most abused word in the industry. People seem to use workflow for all of these things: #1 Business process modeling and automating platforms - Camunda, BPMN, jBPM etc. #2 Assembling together software functions (also called tasks, sometimes) for getting an output data - Airflow, Cadence, CWL is related too. #3 Moving one entity (bug, document, deal) through different stages of a state machine, until the entity reaches an end state - Sharepoint workflow, Zoho Orchestly, Comala workflows #4 Orchestrating several apps together for automating something, usually as reactions for events - Zapier When developers talk about workflows, it's mostly about #2. When business people talk about workflows it's mostly about #1. When managers/leads of a team talk about workflows it's mostly about #3. We should accept that Workflow as a word is too ambiguous, and come up with what kind of workflow category the tool belongs to, before branding it as an "one stop solution for all things workflow".
- zekrioca 7y ago
  Couldn't agree more. I do work with workflows (mostly #2 and #4), but in the beginning of my journey I had problems understanding what researchers were actually trying to tell me, and I always got confused. I suspect that they themselves do not fully understand how ambiguous this word can be..
hhas01 7y ago
“we seem to be talking about "Scientific Computing Workflow" or some sort of data pipeline” So, more along the lines of https://en.wikipedia.org/wiki/Dataflow_programming https://en.wikipedia.org/wiki/Dataflow_programming. A good problem space to bring declarative/functional/logic/concatenative concepts and tools, IMO, as opposed to the ubiquitous “hit it with a rock” of the usual C* suspects. Be great to see real progress here, but my bullshit detector is immediately dialed to MAX. The great problem with “workflow systems” is that they very easily disappear up their own ass: “Potemkin solutions” that solve every problem except the customer’s own. (See: https://thedailywtf.com/articles/The_Inner-Platform_Effect https://thedailywtf.com/articles/The_Inner-Platform_Effect). “Enterprise workflows” in particular I think are just a way for indolent code monkeys to shirk learning how the business works for themselves. Looks all very impressive without ever solving shit. .. FWIW I do think Alan Kay was onto something with his Nile strategy (https://news.ycombinator.com/item?id=10535364 https://news.ycombinator.com/item?id=10535364). Pity there wasn’t followthrough (bit of a mayfly, Kay). Ditto Iverson and APL, although APL’s sidelining was more down to a nascent Programming Profession redefining Programming™ to be what they do for users rather than what users (mathematicians in this case) do for themselves.
- tetron 7y ago
  CWL is a declarative/functional language for describing how to execute command line tools (staging input files, mapping arguments, collecting outputs) and how to connect the outputs of one tool to the inputs of the next. It is HPC and cloud agnostic. The same workflow description can run on a laptop or on 1000 cloud compute nodes. Lots of people are already using it to solve their problems, for some examples see https://github.com/search?q=extension%3Acwl+cwlVersion https://github.com/search?q=extension%3Acwl+cwlVersion
  - hhas01 7y ago
    Quickly skimming the CWL User Guide, calling that a “language” is a huge stretch. Looks like writing a bunch of YAML config files which let you write more YAML files to construct a UNIX pipeline and run it. In which case, why didn’t you just use bash? Yeah, I know bash stinks. But this is not my fist rodeo* so I’m struggling here to see how CWL stinks less. See again: Inner-Platform Effect, Greenspun’s Tenth Rule. -- * i.e. I already know how easily “workflow engines” go up their own arse because I’ve done it myself. And CWL fails the same sniff test. Not a good start.

by fowl2 7y ago

oeuviz 7y ago

Fully agree. It seems nobody understands it in the sense of Business Processes/Workflow modelling anymore and it is arbitrary used for "something to do".

liotier 7y ago
I did expect business process modeling... I was confused !

lewisjoe 7y ago

Yes, "Workflow" has to be one of the most abused word in the industry. People seem to use workflow for all of these things: #1 Business process modeling and automating platforms - Camunda, BPMN, jBPM etc. #2 Assembling together software functions (also called tasks, sometimes) for getting an output data - Airflow, Cadence, CWL is related too. #3 Moving one entity (bug, document, deal) through different stages of a state machine, until the entity reaches an end state - Sharepoint workflow, Zoho Orchestly, Comala workflows #4 Orchestrating several apps together for automating something, usually as reactions for events - Zapier When developers talk about workflows, it's mostly about #2. When business people talk about workflows it's mostly about #1. When managers/leads of a team talk about workflows it's mostly about #3. We should accept that Workflow as a word is too ambiguous, and come up with what kind of workflow category the tool belongs to, before branding it as an "one stop solution for all things workflow".

zekrioca 7y ago
Couldn't agree more. I do work with workflows (mostly #2 and #4), but in the beginning of my journey I had problems understanding what researchers were actually trying to tell me, and I always got confused. I suspect that they themselves do not fully understand how ambiguous this word can be..

hhas01 7y ago

“we seem to be talking about "Scientific Computing Workflow" or some sort of data pipeline” So, more along the lines of https://en.wikipedia.org/wiki/Dataflow_programming https://en.wikipedia.org/wiki/Dataflow_programming. A good problem space to bring declarative/functional/logic/concatenative concepts and tools, IMO, as opposed to the ubiquitous “hit it with a rock” of the usual C* suspects. Be great to see real progress here, but my bullshit detector is immediately dialed to MAX. The great problem with “workflow systems” is that they very easily disappear up their own ass: “Potemkin solutions” that solve every problem except the customer’s own. (See: https://thedailywtf.com/articles/The_Inner-Platform_Effect https://thedailywtf.com/articles/The_Inner-Platform_Effect). “Enterprise workflows” in particular I think are just a way for indolent code monkeys to shirk learning how the business works for themselves. Looks all very impressive without ever solving shit. .. FWIW I do think Alan Kay was onto something with his Nile strategy (https://news.ycombinator.com/item?id=10535364 https://news.ycombinator.com/item?id=10535364). Pity there wasn’t followthrough (bit of a mayfly, Kay). Ditto Iverson and APL, although APL’s sidelining was more down to a nascent Programming Profession redefining Programming™ to be what they do for users rather than what users (mathematicians in this case) do for themselves.

tetron 7y ago
CWL is a declarative/functional language for describing how to execute command line tools (staging input files, mapping arguments, collecting outputs) and how to connect the outputs of one tool to the inputs of the next. It is HPC and cloud agnostic. The same workflow description can run on a laptop or on 1000 cloud compute nodes. Lots of people are already using it to solve their problems, for some examples see https://github.com/search?q=extension%3Acwl+cwlVersion https://github.com/search?q=extension%3Acwl+cwlVersion
- hhas01 7y ago
  Quickly skimming the CWL User Guide, calling that a “language” is a huge stretch. Looks like writing a bunch of YAML config files which let you write more YAML files to construct a UNIX pipeline and run it. In which case, why didn’t you just use bash? Yeah, I know bash stinks. But this is not my fist rodeo* so I’m struggling here to see how CWL stinks less. See again: Inner-Platform Effect, Greenspun’s Tenth Rule. -- * i.e. I already know how easily “workflow engines” go up their own arse because I’ve done it myself. And CWL fails the same sniff test. Not a good start.