4 ms·

If CI builds were defined in a programming language that provided clear semantics, abstraction and modularity then porting to a new build host would be 10x easi

by tome 6y ago

If CI builds were defined in a programming language that provided clear semantics, abstraction and modularity then porting to a new build host would be 10x easier. It would be an "annoying" or "fiddly" level of work rather than a time sink costing hours and hours.

brodock 6y ago
considering how similar many CI providers are in terms of how you write your tests, I would say that this is either very achievable or not that worth it. you already can do a lot by writing a bunch of scripts that will be executed on a CI, and just fiddle with differences in like metadata/artifact/how you declare your containers. and while you are there, you can generate the code CI definitions for more than one CI provider (think of it like "multi-cloud" but for CI approach.
chubot 6y ago
Yup, that's where I want to take the Oil shell: https://news.ycombinator.com/item?id=25340456 https://news.ycombinator.com/item?id=25340456 Continuous builds are already written in YAML + shell. Platforms like Github Actions and Travis CI basically use YAML as a really bad syntax for shell scripts (with their own if statements!), and then embed more shell as strings. So Oil should basically generate JSON like this: https://news.ycombinator.com/item?id=24945449 https://news.ycombinator.com/item?id=24945449 and allow you to embed unevaluated (but parsed) literal shell blocks. So I'm adding the missing "declarative" part to shell. That alone gets you pretty far. Although there are some more language semantics that most builds need, like coarse-grained dependencies, ephemeral or persistent state, etc. gg is a related "multi cloud" system that's very interesting: https://lobste.rs/s/virbxa/papers_i_love_gg https://lobste.rs/s/virbxa/papers_i_love_gg I'm writing a blog post about this right now! ---- An improved shell is the natural language for this problem because shell is for gluing tasks together. It's the language of heterogeneity. I view Make as a form of shell with dependencies, and Make has already been parallelized and distributed by many systems going back many years.

by tome 6y ago

brodock 6y ago

considering how similar many CI providers are in terms of how you write your tests, I would say that this is either very achievable or not that worth it. you already can do a lot by writing a bunch of scripts that will be executed on a CI, and just fiddle with differences in like metadata/artifact/how you declare your containers. and while you are there, you can generate the code CI definitions for more than one CI provider (think of it like "multi-cloud" but for CI approach.

chubot 6y ago

Yup, that's where I want to take the Oil shell: https://news.ycombinator.com/item?id=25340456 https://news.ycombinator.com/item?id=25340456 Continuous builds are already written in YAML + shell. Platforms like Github Actions and Travis CI basically use YAML as a really bad syntax for shell scripts (with their own if statements!), and then embed more shell as strings. So Oil should basically generate JSON like this: https://news.ycombinator.com/item?id=24945449 https://news.ycombinator.com/item?id=24945449 and allow you to embed unevaluated (but parsed) literal shell blocks. So I'm adding the missing "declarative" part to shell. That alone gets you pretty far. Although there are some more language semantics that most builds need, like coarse-grained dependencies, ephemeral or persistent state, etc. gg is a related "multi cloud" system that's very interesting: https://lobste.rs/s/virbxa/papers_i_love_gg https://lobste.rs/s/virbxa/papers_i_love_gg I'm writing a blog post about this right now! ---- An improved shell is the natural language for this problem because shell is for gluing tasks together. It's the language of heterogeneity. I view Make as a form of shell with dependencies, and Make has already been parallelized and distributed by many systems going back many years.