5 ms·

Workflow languages are programming languages. I'd be really interested to know if 1) there's a textual representation of the visual workflow design that you can

Workflow languages are programming languages. I'd be really interested to know if 1) there's a textual representation of the visual workflow design that you can easily export/import, and 2) the syntax and semantics of the language. I've seen a _lot_ of visual workflow editors which have a tendency towards limited expressiveness; there seems to be a major disconnect between the worlds of workflow authoring and programming. I noticed that they provide the ability to specify a Dockerfile which contains the necessary facilities to run arbitrary code. But I can't help but think there has to be a middle ground between the two. I've written about this in the past, arguing that applying concepts from traditional programming language theory (in particular functional programming) to the design of workflow languages can be fruitful. https://www.pmkelly.net/publications/wage2008.pdf https://www.pmkelly.net/publications/wage2008.pdf https://www.pmkelly.net/publications/thesis.pdf https://www.pmkelly.net/publications/thesis.pdf

gbaygon 8y ago
There is a mention in the features page* about it: action "Deploy to Production" { needs = "Provision Database" uses = "actions/aws/ec2" runs = "aws deploy --prod" } *see "Configure as code" in https://github.com/features/actions https://github.com/features/actions
- peterkelly 8y ago
  It's common among workflow languages to represent everything as a graph. What I think is more appropriate is an expression-like approach, where the relationships between the different actions are implicit in the syntax. For example, if you have a graph like the following: node A node B node C node D edge B -> A edge C -> A edge D -> B edge D -> C you could write that as an expression like this: A (B D) (C D) In the action example, the "needs" field is a dependency on the task, or in other words a subexpression. If there are multiple independent expressions that all depend on that, you could use something like let/letrec to assign the result of "Provision Database" to a variable, and then reference that from other expressions. Basically, the action syntax they have is like writing an abstract syntax tree where every node explicitly lists its dependencies. An expression is a much more compact form and the subexpressions are implicit in the AST produced by the parser; this is the approach used by most programming languages. But for reasons I still don't fully understand, workflow languages (which I consider to be a specific class of programming language) don't seem to adopt this more compact representation.
  - curun1r 8y ago
    I think it really depends on your audience. We've recently been working on a workflow tool for people significantly less technical than Github's users and we started out with an approach somewhat similar to the one you outline. We developed a compact DSL that maps almost 1-1 with the UI we wanted to build and inferred the dependency graph from the preconditions set and data used in each action. The thinking was that there was no sense in explicit ordering since it could only slow down execution and introduce errors and that allowing preconditions would create a declarative way to ensure "after" behavior when necessary. But what we found in user testing is that our users kept wanting to manually re-order actions and were really uncomfortable with the system just "figuring out" execution order on its own. We had to introduce the concept of UI-only ordering (the backend still tries as hard as possible to execute in parallel without violating the dependency DAG) to give them the illusion of control. But our users aren't programmers, so Github might have bit more leeway to push complex topics like these onto users.
    - peterkelly 8y ago
      I definitely agree about the audience. For non-programmers, the visual approach is very attractive, and provides an intuitive way to display the workflow in a manner that is easy to make sense of. For programmers (i.e. pretty much everyone that uses GitHub), I think a more DSL-like approach would be appropriate, though this doesn't preclude a visual editor as an alternative. As gbaygon mentioned, they do have a DSL, but I think the approach is suboptimal. In a project I'm working on at the moment (which extends the work from my thesis), we're targeting two audiences - programmers write the workflows, but "business" people can see a visualisation. Essentially we take the Scheme code that comprises the workflow and render it as a graph, similar to BPMN. I think that's a nice approach when you have people on staff who have the necessary programming skills and are working alongside non-technical people. But that doesn't apply in every situation, so visual design can be useful when you don't have experienced programmers creating the workflows.
      - simplify 8y ago
        I'm an experienced programmer and I want effective visualizations. I strongly believe that domain-specific visualizations are the way forward to real progress, even if we haven't created a practical one yet.
plumeria 8y ago
Checkout AWS States Language: https://docs.aws.amazon.com/step-functions/latest/dg/concepts-amazon-states-language.html https://docs.aws.amazon.com/step-functions/latest/dg/concept...
preya2k 8y ago
GitHubs language for the new Actions feature is based on HCL, which is a superset of JSON (adding functionality like comments and other stuff). https://github.com/hashicorp/hcl https://github.com/hashicorp/hcl
- scrollaway 8y ago
  Oh god. hcl is one of the worst parts of terraform. I really wonder what other declarative languages were considered for this because it feels like a very weird base to start with :/
  - brazzledazzle 8y ago
    Is it possible some of your perception is shaped by Terraform’s implementation? I’m not a huge fan of json/yaml/etc as a DSL but Terraform makes/made some choices that leave me empty inside. But only after I read the original issue with hundreds of comments that was then migrated by a bot to a new repo issue which is then closed after hundreds more comments because a more recent issue that’s more specific is now around and then that one is closed after dozens of comments because there’s a roll up issue.
    - scrollaway 8y ago
      Maybe. On the other hand a lot of terraform's implementation has been shaped by how they themselves designed hcl, right? Things like multi-nesting function calls inside quotes and what not. All the dollar signs. I can't even tell if that's terraform or HCL, to be honest.
  - skybrian 8y ago
    Based on the README, hcl looks like a small and fairly sane extension to JSON. What do you dislike about it?
    - scrollaway 8y ago
      Here's an example of a nested function call from one of my infrastructure files: public_key = "${file("${path.root}/../ssh/id_rsa.pub")}" Is this terraform's implementation, or hcl's? HCL has to implement that syntax somehow, right?

gbaygon 8y ago

There is a mention in the features page* about it: action "Deploy to Production" { needs = "Provision Database" uses = "actions/aws/ec2" runs = "aws deploy --prod" } *see "Configure as code" in https://github.com/features/actions https://github.com/features/actions

peterkelly 8y ago
It's common among workflow languages to represent everything as a graph. What I think is more appropriate is an expression-like approach, where the relationships between the different actions are implicit in the syntax. For example, if you have a graph like the following: node A node B node C node D edge B -> A edge C -> A edge D -> B edge D -> C you could write that as an expression like this: A (B D) (C D) In the action example, the "needs" field is a dependency on the task, or in other words a subexpression. If there are multiple independent expressions that all depend on that, you could use something like let/letrec to assign the result of "Provision Database" to a variable, and then reference that from other expressions. Basically, the action syntax they have is like writing an abstract syntax tree where every node explicitly lists its dependencies. An expression is a much more compact form and the subexpressions are implicit in the AST produced by the parser; this is the approach used by most programming languages. But for reasons I still don't fully understand, workflow languages (which I consider to be a specific class of programming language) don't seem to adopt this more compact representation.
- curun1r 8y ago
  I think it really depends on your audience. We've recently been working on a workflow tool for people significantly less technical than Github's users and we started out with an approach somewhat similar to the one you outline. We developed a compact DSL that maps almost 1-1 with the UI we wanted to build and inferred the dependency graph from the preconditions set and data used in each action. The thinking was that there was no sense in explicit ordering since it could only slow down execution and introduce errors and that allowing preconditions would create a declarative way to ensure "after" behavior when necessary. But what we found in user testing is that our users kept wanting to manually re-order actions and were really uncomfortable with the system just "figuring out" execution order on its own. We had to introduce the concept of UI-only ordering (the backend still tries as hard as possible to execute in parallel without violating the dependency DAG) to give them the illusion of control. But our users aren't programmers, so Github might have bit more leeway to push complex topics like these onto users.
  - peterkelly 8y ago
    I definitely agree about the audience. For non-programmers, the visual approach is very attractive, and provides an intuitive way to display the workflow in a manner that is easy to make sense of. For programmers (i.e. pretty much everyone that uses GitHub), I think a more DSL-like approach would be appropriate, though this doesn't preclude a visual editor as an alternative. As gbaygon mentioned, they do have a DSL, but I think the approach is suboptimal. In a project I'm working on at the moment (which extends the work from my thesis), we're targeting two audiences - programmers write the workflows, but "business" people can see a visualisation. Essentially we take the Scheme code that comprises the workflow and render it as a graph, similar to BPMN. I think that's a nice approach when you have people on staff who have the necessary programming skills and are working alongside non-technical people. But that doesn't apply in every situation, so visual design can be useful when you don't have experienced programmers creating the workflows.
    - simplify 8y ago
      I'm an experienced programmer and I want effective visualizations. I strongly believe that domain-specific visualizations are the way forward to real progress, even if we haven't created a practical one yet.

plumeria 8y ago

Checkout AWS States Language: https://docs.aws.amazon.com/step-functions/latest/dg/concepts-amazon-states-language.html https://docs.aws.amazon.com/step-functions/latest/dg/concept...

preya2k 8y ago

GitHubs language for the new Actions feature is based on HCL, which is a superset of JSON (adding functionality like comments and other stuff). https://github.com/hashicorp/hcl https://github.com/hashicorp/hcl

scrollaway 8y ago
Oh god. hcl is one of the worst parts of terraform. I really wonder what other declarative languages were considered for this because it feels like a very weird base to start with :/
- brazzledazzle 8y ago
  Is it possible some of your perception is shaped by Terraform’s implementation? I’m not a huge fan of json/yaml/etc as a DSL but Terraform makes/made some choices that leave me empty inside. But only after I read the original issue with hundreds of comments that was then migrated by a bot to a new repo issue which is then closed after hundreds more comments because a more recent issue that’s more specific is now around and then that one is closed after dozens of comments because there’s a roll up issue.
  - scrollaway 8y ago
    Maybe. On the other hand a lot of terraform's implementation has been shaped by how they themselves designed hcl, right? Things like multi-nesting function calls inside quotes and what not. All the dollar signs. I can't even tell if that's terraform or HCL, to be honest.
- skybrian 8y ago
  Based on the README, hcl looks like a small and fairly sane extension to JSON. What do you dislike about it?
  - scrollaway 8y ago
    Here's an example of a nested function call from one of my infrastructure files: public_key = "${file("${path.root}/../ssh/id_rsa.pub")}" Is this terraform's implementation, or hcl's? HCL has to implement that syntax somehow, right?