13 ms·

I'm confused, a DAG is a model for the relationships between data. What makes this a separate feature for the product? And it's pretty fundamental to the proble

by qppo 6y ago

I'm confused, a DAG is a model for the relationships between data. What makes this a separate feature for the product? And it's pretty fundamental to the problem you try to solve, usually, so what were they using before, a list? Tree?

detaro 6y ago
Which part of the explanation of what they mean in the article is unclear?
- corndoge 6y ago
  They're sort of coopting the name of a datastructure and using it as if it were a product name which is a bit weird
- qppo 6y ago
  The article is clear on what they're doing. My confusion is to whether this is a feature announcement or a technical blog on the implementation details.
chrisseaton 6y ago
> I'm confused, a DAG is a model for the relationships between data. Here they're talking about a DAG as a model for the relationships between computation rather than data. Many systems that model computation use DAGs. For example it's how many compilers understand your code when you compile it.
- throwaway894345 6y ago
  > Here they're talking about ... computation rather than data. The lisp people would like a word... In seriousness, computation is data, and this is perhaps especially obvious in a CI context where your “computation” is increasingly bash scripts in some string field inside of a YAML file.
  - chrisseaton 6y ago
    > The lisp people would like a word... Why do you feel the need to say things in such a snarky way as this, as if I'm ignorant? That was the point - that computation can also be represented in a 'data' structure like this. I already gave that exact example - a compiler representing computation as data.
    - qppo 6y ago
      I think they're replying that way because you seem to miss my point, I didn't think I needed to say that "the operations are the data" explicitly. My confusion was not on what they were doing or why they chose to use a DAG, but why they're presenting it this way and why they weren't using a DAG to begin with. Like I've written task runners in various forms before. They've always been based on DAGs and topological sorting from the beginning. I wouldn't advertise it as such because the choice of model is obvious to me, I'd be more interested in reading about alternatives that provide other benefits.
    - throwaway894345 6y ago
      Whoa. I wasn't trying to be snarky; if anything it was intended to be a humorous jab at how "computation is data" is such a thing in the lisp community. I agree that your example illustrates the "computation is data" concept; I was just reconciling that with the apparent contradiction ("computation rather than data") earlier in your post. I wasn't trying to call "gotcha" on you or anything; sometimes I say things that are unclear and I appreciate it when others help me clarify.
gaff33 6y ago
Historically gitlab had a stages system: "Do all your builds, then all your tests, then all your deployments"
wolfgang42 6y ago
They were using a list-of-sets; the CI pipeline is composed of stages which run in series, each of which runs a set of jobs in parallel. The new dependency DAG allows jobs to explicitly declare dependencies on each other and avoid waiting at stage boundaries.
tr352 6y ago
Agreed. First sentence: "A Directed Acyclic Graph will let you run pipeline steps out of order." That's simply false.
- defen 6y ago
  Interpret "order" as "the order in which the steps are defined in your YAML file", and it's simply true.
  - deleted 6y ago
    [deleted]
  - tr352 6y ago
    I'm nitpicking but my point is that a DAG doesn't let you run anything. It might represent something, though, such as a bunch of tasks and a "must be executed before" relation. It's sloppy, imprecise use of language.
    - BorisTheBrave 6y ago
      By that logic, no code lets you run anything, it just "represents" a program that actually runs. That's just pedantry.
      - tr352 6y ago
        Code runs and possibly lets you run something (for example in case of an OS or interpreter). I don't see a problem there.

by qppo 6y ago

detaro 6y ago

Which part of the explanation of what they mean in the article is unclear?

corndoge 6y ago
They're sort of coopting the name of a datastructure and using it as if it were a product name which is a bit weird
qppo 6y ago
The article is clear on what they're doing. My confusion is to whether this is a feature announcement or a technical blog on the implementation details.

chrisseaton 6y ago

> I'm confused, a DAG is a model for the relationships between data. Here they're talking about a DAG as a model for the relationships between computation rather than data. Many systems that model computation use DAGs. For example it's how many compilers understand your code when you compile it.

throwaway894345 6y ago
> Here they're talking about ... computation rather than data. The lisp people would like a word... In seriousness, computation is data, and this is perhaps especially obvious in a CI context where your “computation” is increasingly bash scripts in some string field inside of a YAML file.
- chrisseaton 6y ago
  > The lisp people would like a word... Why do you feel the need to say things in such a snarky way as this, as if I'm ignorant? That was the point - that computation can also be represented in a 'data' structure like this. I already gave that exact example - a compiler representing computation as data.
  - qppo 6y ago
    I think they're replying that way because you seem to miss my point, I didn't think I needed to say that "the operations are the data" explicitly. My confusion was not on what they were doing or why they chose to use a DAG, but why they're presenting it this way and why they weren't using a DAG to begin with. Like I've written task runners in various forms before. They've always been based on DAGs and topological sorting from the beginning. I wouldn't advertise it as such because the choice of model is obvious to me, I'd be more interested in reading about alternatives that provide other benefits.
  - throwaway894345 6y ago
    Whoa. I wasn't trying to be snarky; if anything it was intended to be a humorous jab at how "computation is data" is such a thing in the lisp community. I agree that your example illustrates the "computation is data" concept; I was just reconciling that with the apparent contradiction ("computation rather than data") earlier in your post. I wasn't trying to call "gotcha" on you or anything; sometimes I say things that are unclear and I appreciate it when others help me clarify.

gaff33 6y ago

Historically gitlab had a stages system: "Do all your builds, then all your tests, then all your deployments"

wolfgang42 6y ago

They were using a list-of-sets; the CI pipeline is composed of stages which run in series, each of which runs a set of jobs in parallel. The new dependency DAG allows jobs to explicitly declare dependencies on each other and avoid waiting at stage boundaries.

tr352 6y ago

Agreed. First sentence: "A Directed Acyclic Graph will let you run pipeline steps out of order." That's simply false.

defen 6y ago
Interpret "order" as "the order in which the steps are defined in your YAML file", and it's simply true.
- deleted 6y ago
  [deleted]
- tr352 6y ago
  I'm nitpicking but my point is that a DAG doesn't let you run anything. It might represent something, though, such as a bunch of tasks and a "must be executed before" relation. It's sloppy, imprecise use of language.
  - BorisTheBrave 6y ago
    By that logic, no code lets you run anything, it just "represents" a program that actually runs. That's just pedantry.
    - tr352 6y ago
      Code runs and possibly lets you run something (for example in case of an OS or interpreter). I don't see a problem there.