15 ms·

Cicada – Open-source cross-platform version of GitHub Actions and Gitlab CI

162 points by microflash 3y ago

danpalmer 3y ago
Having built CI/CD pipelines in CircleCI, Jenkins, GitLab CI, Concourse, Semaphore, and GitHub Actions, I'm not entirely sure what this offers, other than being FOSS. That's not to say that it being FOSS is not valuable – for some users it most certainly is – but that I can't see another way that this would be valuable. The config language is nice, but honestly I've never been bothered by YAML for pipelines. I want my pipelines to be declarative anyway. I'm a little disappointed that Cicada didn't go with one of the existing options such as Jsonnet, Cue, Dhall, etc. Lastly, there's a feature I've always found useful and in many cases necessary, that many CI systems don't have – enforced serial execution. When you're doing continuous delivery it's often critical to make sure that only one release is going out at a time and that the version being released strictly increases. I've seen outages because of release jobs racing and an unintended downgrade happening. This requires CI system support to achieve. Last time I checked, Jenkins, Concourse, and GitHub Actions all provided mechanisms for this, Semaphore may have as well. Circle and GitLab did not (the former after endless discussion with our account manager over it!) and I found it hard to trust the platforms as a result. Cicada does not appear to have this, which is a shame. It suggests to me a lack of hands-on production experience with continuous delivery. Arguably this is not a CD system, but there's no reason why a CI system shouldn't be CD as well, it's not until a much larger team/product size that a dedicated CD system becomes truly necessary and they take a lot of work to set up well.
- digi59404 3y ago
  Maybe I’m misunderstanding? For GitLab this should be doable by two different ways. The first is merge trains, which merges requests one by one to prevent this exact outcome. You just have to force all deployments to be done via a merge request. That’s the downside. The second being forcing a GitLab Runner to run one job at a time. Tag it as “deployer” then ensure all deployment jobs are marked as “deployer”. That runner will pick up deployment jobs one by one in order of first creation.
  - danpalmer 3y ago
    I think merge trains weren't a feature when I worked with GitLab, it was a while ago. This does certainly solve it, but at the cost of quite a different process. If what you're optimising for is time to release, merge trains add an overhead. At some point that overhead is worth it, but it depends on the team/product/etc. Having just one runner be the deployer is an option too. I think we used hosted runners so not sure if this is possible in that setup? This would also make pipelines harder to optimise. Often there are many parts in pipelines that are safe to do in parallel, and only a few "critical sections" around which you want locking. This would solve simultaneous releases, but not the general case of the problem (which at least Jenkins and GitHub Actions manage ok). GitLab always felt to me like Travis++, whereas systems developed later felt like they were built on fundamentally better primitives. Jenkins is a weird one because it has all of the features, can do all of these things, really quite well in many cases, but has a pretty bad developer experience and required a lot of maintenance to run a performant and secure install.
  - tsak 3y ago
    There's also a third way called resource groups [1]. We use that to ensure that we only run the newest job if we have multiple deployment jobs waiting for execution. This way even if we have multiple pipelines racing each other, only the last deployment job wins. [1] https://docs.gitlab.com/ee/ci/resource_groups/index.html https://docs.gitlab.com/ee/ci/resource_groups/index.html
    - imp0cat 3y ago
      Yeah, we use that instead of dedicated runners, too. But it is a relatively new Gitlab concept, isn't it?
      - tsak 3y ago
        We've been using it for a year, as we often had the problem that somebody would push an update after an initial publish of a branch, not stopping the first branch and then two or more deploy jobs racing each other with unpredictable results.
- matrss 3y ago
  > I want my pipelines to be declarative anyway. Is there any CI system that allows one to write declarative pipelines? What would that even mean? GitHub Actions, GitLab Pipelines, etc. are all effectively just shell scripts disguised as more or less verbose yaml, with some trigger conditions added. I guess nix's hydra is the closest to a declarative CI system that exists, because nix does the hard work of abstracting the imperative build steps into a declarative interface. Even then, if you want to do anything outside of nix derivations you would be writing something imperative.
  - pydry 3y ago
    it's github actions but you keep it short and put everything you can in a script that gets shelled out to.
    - matrss 3y ago
      So, only use the CI system for it's dispatch/scheduling part and put the imperative steps into a dedicated script? Reasonable, and what I have done with some GitLab Pipelines as well, but I don't see how that would be considered more "declarative".
    - summarity 3y ago
      Buildkite was once that.
  - tonyhb 3y ago
    Check out https://dagger.io/ https://dagger.io/. Write declarative pipelines in code, reproducibly run anywhere.
  - danpalmer 3y ago
    I'm not looking for literally everything to be declarative, but ideally each snippet of shell script would be entirely independent and run effectively reproducibly. GitHub/GitLab do this well enough, although not perfectly. Concourse does this really well. Bazel is also great at this.
    - 3y ago
- dosisod 3y ago
  Creator of Cicada here. Thank you for the feedback! I've mentioned this in a few threads already, but the reason for making a new DSL for writing the workflows is that YAML makes it hard/cumbersome to express more complex workflows. Using a programming language though gives you more control over how your workflows execute. While there are already plenty of tools that use existing programming languages (ie, Python/TypeScript) to configure workflows, having a custom DSL allows you to make some of the more abstract terms like caching, conditional execution, permissions, etc. more explicit. To your last point, I have experience with using CD in production, but not to the scale where I have builds stepping over each other and causing issues. I agree that serial builds are important in this case, and is something that I will need to look into (conceptually it sounds pretty simple).
  - melezhik 3y ago
    You might want to check out the SparrowCI - it has combination of yaml AND standard programming languages ( Python/Ruby/GoLang/Powershell/Bash/Raku) giving you the best of two worlds - declarative style and extremely flexible flows when required
  - danpalmer 3y ago
    Awesome! I think serial builds should be fairly simple to implement if you can tie it back to a transaction in a data store. I get the DSL desire, and I feel I've already lost the "YAML is fine" battle elsewhere so that's not a problem. I think a language like jsonnet or cue would have been a better choice simply because they don't involve users learning a new language or you from implementing a new language. Both would have allowed plugging in your own standard library of functions and abstractions.
- deleted 3y ago
  [deleted]
skinner927 3y ago
This project looks interesting, I very much like how secrets are handled, but here’s 3 reasons why, given what I can gather from the docs, I can’t use it right now. 1. It doesn’t seem it’s possible to include other .ci files? I have multiple projects that use the same ci config with their own augments and Cicada seemingly won’t work with that flow? 2. Self hosted non-docker runners are Python3.11. Some of us (albeit few of us) don’t have the luxury of being able to abandon ancient OS targets. 3. Doesn’t seem git.push allows branch to specified as “$DEFAULT_BRANCH” macro (or similar). Some projects use master, some use main, some use gold, whatever, it would be nice to not have to know. The example CI repo is no more than a “hello world”. I don’t think people with simple CI requirements are interesting in switching from what they already have. Your target audience is likely someone like me who maintains 10k+ lines (merged) of GitLab YAML and wants to get out. I would be more encouraged to look deeper into this project if it could show me the value it adds, because right now it just seems like a different YAML that I’ll eventually loathe too. Very neat project, I hope to see it mature.
- snapplebobapple 3y ago
  It also seems to require github credentials instead of just supporting saml/ldap/local accounts and I don't see a way to make it work with over git providers (i.e. gitea) so not sure where the home gamer market would use this either? If I'm ponying up private github repositories already why wouldn't I just pony up slightly more to use their system? maybe I'm just missing it (I am, after all, a noob just scraping by on beginning to learn git and using it to manage a few things).
  - dosisod 3y ago
    Creator of Cicada here. Currently I am using GitHub SSO to reduce spam for the logins. When installing Cicada locally though you need to create an admin account, though there is no way (in the UI) to create new local users. Adding more sign-in methods (ie Gitlab, Bitbucket) is definitely on my radar.
- dosisod 3y ago
  Creator of Cicada here. Thank you for these questions! I'll try and answer them all: 0: Secrets are stored using Vault. Read this commit message [1] for a full breakdown. 1: Currently you cannot include other .ci files. I have been in feature creep mode for months now, and I've been forcing myself to stop adding features and start talking to users. The goal is to make Cicada more or less a general purpose programming language, but the first step is making it work well for defining CI/CD workflows. 2: If it is necessary, I could back-port the self-hosted runners to Python 3.10/3.9 or earlier. And, since the runner interface just uses websockets, I (or someone else) could make a runner in a different language, ie Rust or Go. 3: There is a "event" global variable that includes info like "event.branch", which is the branch that was pushed, but it does not include stuff like the default branch. Currently you could do `on git.push where event.branch is "main"`, but something like "event.branch is event.default_branch" would be even better. I'll work on adding that. The value add currently is that Cicada is FOSS, platform agnostic (works with GitHub/Gitlab), and uses a language that consolidates the workflows and scripts into one manageable file format. Existing CI systems are already packed with features that people expect, so the current struggle is catching up to this and then adding more on top of that. I'm trying to focus on what sets Cicada apart: That it gives you more control over your workflows, while being expressive and easier to manage then YAML and shell scripts. [1]: https://github.com/Cicada-Software/cicada/commit/2659f79b5009a7a53ae28b2c465f956cbfd81127 https://github.com/Cicada-Software/cicada/commit/2659f79b500...
spankalee 3y ago
This is very light on details. Does it actually allow you to run your GitHub Actions on your own servers?
- capableweb 3y ago
  This is the syntax of the pipeline it seems: https://github.com/Cicada-Software/demo/blob/main/hello_world.ci https://github.com/Cicada-Software/demo/blob/main/hello_worl... Short answer: No
  - deleted 3y ago
    [deleted]
- dboreham 3y ago
  See comment above for how you can do that.
barkingcat 3y ago
There is a collision in names: Cicada, this CI tool, uses a DSL (domain specific language) to write configuration, and this DSL is referred to as "Cicada language", and blasted in marketing copy as a "real programming language" on https://cicada.sh/ https://cicada.sh/ . The Cicada DSL is documented in https://cicada.sh/docs/ci-lang/index.html https://cicada.sh/docs/ci-lang/index.html However, this is a completely different language from Cicada language, a programming language and theorem prover hosted at https://cicada-lang.org/ https://cicada-lang.org/ and https://github.com/cicada-lang/cicada https://github.com/cicada-lang/cicada . This name collision is very confusing, and I wonder why Cicada the CI tool didn't just stick to python, since it is also a "real programming language."
- deleted 3y ago
  [deleted]
- bogwog 3y ago
  I didn't know about that other Cicada language, but the part about using a DSL for this does seem unnecessary to me. I haven't dived into the docs at all, so if anyone has I'd be curious to know what this DSL offers over Python, Starlark, or any other existing programming language? Especially since, as far as I can tell, this is built on Python. So you already have a working Python installation at least.
  - dosisod 3y ago
    Creator of Cicada here. I created a new DSL because I wanted a more declarative, expressive, yet capable file format, something like Dockerfile but with programming capabilities. In terms of "just running commands", Python (or any other language) is just as good if not better. Where Cicada shines though is that it makes higher level concepts like conditional execution and caching front end center, so you can describe more with less lines. The DSL itself isn't very mature at the moment, but I plan on adding stuff like allow/deny capabilities and more in the future.
dbingham 3y ago
This is a misleading title. Being yet another CI tool is not the same thing as being a cross platform version of Github Actions and Gitlab CI. A cross platform version of Github Actions would allow you to run your actions using your own tooling.
- dboreham 3y ago
  Also this exists: https://github.com/nektos/act https://github.com/nektos/act which is conveniently packaged in a GH-a-like system by: https://gitea.com/gitea/act_runner https://gitea.com/gitea/act_runner in https://github.com/go-gitea https://github.com/go-gitea
  - tao_at_garden 3y ago
    And there's us, https://garden.io https://garden.io, that does totally portable CI pipelines with caching from builds to tests: https://github.com/garden-io/garden https://github.com/garden-io/garden I made an irreverent short on why your CI pipelines ought to be portable and runnable locally. Number one reason? Give your developers their time (and sanity) back. Pushing to Git in the inner loop disrupts flow and shatters attention. https://www.youtube.com/watch?v=JFnan6s2cDg https://www.youtube.com/watch?v=JFnan6s2cDg
    - dosisod 3y ago
      Nice product! I haven't heard of Garden until now. Adding a CLI for Cicada has been on my todo list for a while, and is long overdue! This should be easy enough to do.
    - eysi 3y ago
      Another Gardener here :) Just to add to what Tao was saying, our pipelines are not only portable but also "smart". Instead of having to specify every step of a job—either in code or config—you instead tell Garden that it's e.g. a build of type container, or a deploy of type Helm, and our plugin system figures out the rest (of course with escape hatches when needed). We also track the files that go "into" each job and cache the results, so the same job never has to run twice if the code doesn't change. So if you have a large distributed system and a change only affects a small part of it, you don't need to re-run everything. It can save _a lot_ of time. Unrelated, but love that the Cicada team created a Treesitter grammar for Neovim!
debarshri 3y ago
Something I have noticed lately, First startup idea that every Devops person starts with is to create a CI/CD project because they think world is inefficient and workflows can be optimized, and then they pivot to other ideas.
- j45 3y ago
  So would that mean there is still room for improvement?
  - debarshri 3y ago
    I think market is very mature for CI/CD, it often boils down to price war. Economics of operating your own CI/CD in org like this one is fairly bad compared to Github actions. Especially, the way Github bundles actions with the other services as a buyer it definitely feels like a better deal than buying best in class CI/CD. Github actions gets the job done. The slow changing nature of pipelines makes then candidate for not being touch once the configuration is set. So, to answer your question, in my opinion - is there a room for improvement - yes, but the value is minuscule for a customer to switch to a different provider in CI/CD is working fairly well.
    - j45 3y ago
      Appreciate that. The economics may appear bad for not using GitHub, but there’s still valid needs for code that isn’t hosted in the cloud. For those cases it is reasonable to see or know what’s out there.
      - debarshri 3y ago
        Fair enough.
- intelVISA 3y ago
  I always thought its ease of creation (few weekends) was the main driver?
- 3y ago
leonheld 3y ago
What I really want is a 'Grand Translator' tool that allows any CI pipeline definition to be translated into any other CI pipeline definition. I want to take any .gitlab-ci.yml and magically translate it to a github workflow, and vice-versa. I know it isn't impossible, but it's a heck lot of work to get it right with all the hidden features behind declarative pipelines.
- ska 3y ago
  If doing this sort of a migration is a real possibility, it's an argument for having the bare minimum logic in the CI tool and as much as you can in your own scripts...
  - dindresto 3y ago
    That's actually what we did for our monorepo. We had a huge Gitlab pipeline of multiple steps and jobs, now it's a single job pipeline that builds, tests and deploys 15 different projects, all powered by Nix and some Python scripts.
  - leonheld 3y ago
    Ugh, yes, but then I fall into the "too complex for bash, too simple for Python" trap. I feel like everything goes to hell the moment someone writes a Python class to deal with /something/ in a build pipeline. For bash, well, we all know where that goes... the other options are simply too obscure for me to invest in. But I agree with your assessment, as much as the classic script: - chmod +x pipeline.sh - ./pipeline.sh makes me want to die.
- sakopov 3y ago
  We need a "terraform" for build pipelines.
  - _joel 3y ago
    oh god, please, no. :) I mean, go for it if you want but I'm not sure why you'd need to maintain so many heterogeous pipelines that would warrant a tool like this.
endigma 3y ago
You should change your name. "Cicada" was also a CI/CD tool recently sunset by AWS after acquiring Fig. I maintain the community fork so I happen to know Amazon still holds a trademark on "Cicada"[1]. We rebranded to "Katoa" but you're going to run into them directly especially if you have any commercial plans. 1: Can't link, you can search "Cicada" and "Amazon" on the USPTO TESS
- joshuaissac 3y ago
  Here it is: https://tsdr.uspto.gov/search.action?sn=98150978 https://tsdr.uspto.gov/search.action?sn=98150978 > online non-downloadable software for operating and managing continuous integration and continuous delivery pipelines in computer programming environments
- deleted 3y ago
  [deleted]
biugbkifcjk 3y ago
I'm going to plug https://onedev.io/ https://onedev.io/ its awesome. Its self hosted and has its own tooling for CI/CD. I feel it doesn't get enough love, but I've been using it for years for my own stuff.
- trescenzi 3y ago
  Oh this looks really neat. I've been slowly hand rolling a home server. Started with Gogs felt that was too heavy, now I'm at cgit + gitolite which is awesome but I'm not sure what I want to do when it comes to CI/CD. This is probably on the scale of Gogs but something about it feels more reasonable lol.
  - biugbkifcjk 3y ago
    There is a bit of a learning curve, but there is for any new platform. The devs are very responsive and looks like they're doing an enterprise version for funding. It uses Agents which run the tasks in the CI/CD jobs.
- deleted 3y ago
  [deleted]
moffkalast 3y ago
Cicada 3301 finally produced something? /s
- thepostman0 3y ago
  Unsigned though
reactordev 3y ago
Mods, can we change the title to reflect what this really is, Open Source CI platform (not GitHub Actions, not GitLab CI). Unless I’m mistaken, this project doesn’t use their flavors of yaml. So calling it a “version of” is a lie.
- deleted 3y ago
  [deleted]
xinayder 3y ago
What's the difference between this and Gitea/Forgejo Runners that are compatible with GitHub Actions?
djha-skin 3y ago
I have logic that needs to be put into the build. I can put it into a DSL or I can put it into a bash script. A bash script can be run on a developer's machine. This a bash script allows the developer to test the build on their own machine without the need for running an entire build through the CI system first. I therefore don't see the need for all that DSL stuff that's designed around not needing a bash script. I still need the batch script for testability purposes. I'm fine with the DSL otherwise; switching from Jenkins to GitHub actions isn't a big deal especially when all you're going to do is run a bash script.
throwawaaarrgh 3y ago
This looks really interesting, but I wish people would instead contribute to WoodpeckerCI. It's a FOSS fork of Drone, and Drone really is the pinnacle of CI for me. Only what you need, no fat, simple, functional, modern, easy. IMHO the only thing Woodpecker really needs right now is help finishing their core implementations of K8s support and support for the big 3 DVCS vendors.
- skinner927 3y ago
  WoodpeckerCI only works with containers. Many of us are not building web apps.
iFire 3y ago
LICENSE AGPL https://github.com/Cicada-Software/cicada/blob/main/LICENSE https://github.com/Cicada-Software/cicada/blob/main/LICENSE I.. wouldn't use to build commercial software.
- Yasuraka 3y ago
  You seem to have no idea what AGPL entails or be maliciously spreading FUD.
  - robertlagrant 3y ago
    Maybe attack the argument, not the person?
thamer 3y ago
If anyone from the project is reading this, the multiple misspellings or typos featured prominently on your website make it look very amateurish (imo). Between "You deserve better then YAML", "autotomatically escaped", and other weird phrasing, this doesn't really inspire confidence.

162 points by microflash 3y ago

Having built CI/CD pipelines in CircleCI, Jenkins, GitLab CI, Concourse, Semaphore, and GitHub Actions, I'm not entirely sure what this offers, other than being FOSS. That's not to say that it being FOSS is not valuable – for some users it most certainly is – but that I can't see another way that this would be valuable. The config language is nice, but honestly I've never been bothered by YAML for pipelines. I want my pipelines to be declarative anyway. I'm a little disappointed that Cicada didn't go with one of the existing options such as Jsonnet, Cue, Dhall, etc. Lastly, there's a feature I've always found useful and in many cases necessary, that many CI systems don't have – enforced serial execution. When you're doing continuous delivery it's often critical to make sure that only one release is going out at a time and that the version being released strictly increases. I've seen outages because of release jobs racing and an unintended downgrade happening. This requires CI system support to achieve. Last time I checked, Jenkins, Concourse, and GitHub Actions all provided mechanisms for this, Semaphore may have as well. Circle and GitLab did not (the former after endless discussion with our account manager over it!) and I found it hard to trust the platforms as a result. Cicada does not appear to have this, which is a shame. It suggests to me a lack of hands-on production experience with continuous delivery. Arguably this is not a CD system, but there's no reason why a CI system shouldn't be CD as well, it's not until a much larger team/product size that a dedicated CD system becomes truly necessary and they take a lot of work to set up well.

digi59404 3y ago
Maybe I’m misunderstanding? For GitLab this should be doable by two different ways. The first is merge trains, which merges requests one by one to prevent this exact outcome. You just have to force all deployments to be done via a merge request. That’s the downside. The second being forcing a GitLab Runner to run one job at a time. Tag it as “deployer” then ensure all deployment jobs are marked as “deployer”. That runner will pick up deployment jobs one by one in order of first creation.
- danpalmer 3y ago
  I think merge trains weren't a feature when I worked with GitLab, it was a while ago. This does certainly solve it, but at the cost of quite a different process. If what you're optimising for is time to release, merge trains add an overhead. At some point that overhead is worth it, but it depends on the team/product/etc. Having just one runner be the deployer is an option too. I think we used hosted runners so not sure if this is possible in that setup? This would also make pipelines harder to optimise. Often there are many parts in pipelines that are safe to do in parallel, and only a few "critical sections" around which you want locking. This would solve simultaneous releases, but not the general case of the problem (which at least Jenkins and GitHub Actions manage ok). GitLab always felt to me like Travis++, whereas systems developed later felt like they were built on fundamentally better primitives. Jenkins is a weird one because it has all of the features, can do all of these things, really quite well in many cases, but has a pretty bad developer experience and required a lot of maintenance to run a performant and secure install.
- tsak 3y ago
  There's also a third way called resource groups [1]. We use that to ensure that we only run the newest job if we have multiple deployment jobs waiting for execution. This way even if we have multiple pipelines racing each other, only the last deployment job wins. [1] https://docs.gitlab.com/ee/ci/resource_groups/index.html https://docs.gitlab.com/ee/ci/resource_groups/index.html
  - imp0cat 3y ago
    Yeah, we use that instead of dedicated runners, too. But it is a relatively new Gitlab concept, isn't it?
    - tsak 3y ago
      We've been using it for a year, as we often had the problem that somebody would push an update after an initial publish of a branch, not stopping the first branch and then two or more deploy jobs racing each other with unpredictable results.
matrss 3y ago
> I want my pipelines to be declarative anyway. Is there any CI system that allows one to write declarative pipelines? What would that even mean? GitHub Actions, GitLab Pipelines, etc. are all effectively just shell scripts disguised as more or less verbose yaml, with some trigger conditions added. I guess nix's hydra is the closest to a declarative CI system that exists, because nix does the hard work of abstracting the imperative build steps into a declarative interface. Even then, if you want to do anything outside of nix derivations you would be writing something imperative.
- pydry 3y ago
  it's github actions but you keep it short and put everything you can in a script that gets shelled out to.
  - matrss 3y ago
    So, only use the CI system for it's dispatch/scheduling part and put the imperative steps into a dedicated script? Reasonable, and what I have done with some GitLab Pipelines as well, but I don't see how that would be considered more "declarative".
  - summarity 3y ago
    Buildkite was once that.
- tonyhb 3y ago
  Check out https://dagger.io/ https://dagger.io/. Write declarative pipelines in code, reproducibly run anywhere.
- danpalmer 3y ago
  I'm not looking for literally everything to be declarative, but ideally each snippet of shell script would be entirely independent and run effectively reproducibly. GitHub/GitLab do this well enough, although not perfectly. Concourse does this really well. Bazel is also great at this.
  - 3y ago
dosisod 3y ago
Creator of Cicada here. Thank you for the feedback! I've mentioned this in a few threads already, but the reason for making a new DSL for writing the workflows is that YAML makes it hard/cumbersome to express more complex workflows. Using a programming language though gives you more control over how your workflows execute. While there are already plenty of tools that use existing programming languages (ie, Python/TypeScript) to configure workflows, having a custom DSL allows you to make some of the more abstract terms like caching, conditional execution, permissions, etc. more explicit. To your last point, I have experience with using CD in production, but not to the scale where I have builds stepping over each other and causing issues. I agree that serial builds are important in this case, and is something that I will need to look into (conceptually it sounds pretty simple).
- melezhik 3y ago
  You might want to check out the SparrowCI - it has combination of yaml AND standard programming languages ( Python/Ruby/GoLang/Powershell/Bash/Raku) giving you the best of two worlds - declarative style and extremely flexible flows when required
- danpalmer 3y ago
  Awesome! I think serial builds should be fairly simple to implement if you can tie it back to a transaction in a data store. I get the DSL desire, and I feel I've already lost the "YAML is fine" battle elsewhere so that's not a problem. I think a language like jsonnet or cue would have been a better choice simply because they don't involve users learning a new language or you from implementing a new language. Both would have allowed plugging in your own standard library of functions and abstractions.
deleted 3y ago
[deleted]

skinner927 3y ago

This project looks interesting, I very much like how secrets are handled, but here’s 3 reasons why, given what I can gather from the docs, I can’t use it right now. 1. It doesn’t seem it’s possible to include other .ci files? I have multiple projects that use the same ci config with their own augments and Cicada seemingly won’t work with that flow? 2. Self hosted non-docker runners are Python3.11. Some of us (albeit few of us) don’t have the luxury of being able to abandon ancient OS targets. 3. Doesn’t seem git.push allows branch to specified as “$DEFAULT_BRANCH” macro (or similar). Some projects use master, some use main, some use gold, whatever, it would be nice to not have to know. The example CI repo is no more than a “hello world”. I don’t think people with simple CI requirements are interesting in switching from what they already have. Your target audience is likely someone like me who maintains 10k+ lines (merged) of GitLab YAML and wants to get out. I would be more encouraged to look deeper into this project if it could show me the value it adds, because right now it just seems like a different YAML that I’ll eventually loathe too. Very neat project, I hope to see it mature.

snapplebobapple 3y ago
It also seems to require github credentials instead of just supporting saml/ldap/local accounts and I don't see a way to make it work with over git providers (i.e. gitea) so not sure where the home gamer market would use this either? If I'm ponying up private github repositories already why wouldn't I just pony up slightly more to use their system? maybe I'm just missing it (I am, after all, a noob just scraping by on beginning to learn git and using it to manage a few things).
- dosisod 3y ago
  Creator of Cicada here. Currently I am using GitHub SSO to reduce spam for the logins. When installing Cicada locally though you need to create an admin account, though there is no way (in the UI) to create new local users. Adding more sign-in methods (ie Gitlab, Bitbucket) is definitely on my radar.
dosisod 3y ago
Creator of Cicada here. Thank you for these questions! I'll try and answer them all: 0: Secrets are stored using Vault. Read this commit message [1] for a full breakdown. 1: Currently you cannot include other .ci files. I have been in feature creep mode for months now, and I've been forcing myself to stop adding features and start talking to users. The goal is to make Cicada more or less a general purpose programming language, but the first step is making it work well for defining CI/CD workflows. 2: If it is necessary, I could back-port the self-hosted runners to Python 3.10/3.9 or earlier. And, since the runner interface just uses websockets, I (or someone else) could make a runner in a different language, ie Rust or Go. 3: There is a "event" global variable that includes info like "event.branch", which is the branch that was pushed, but it does not include stuff like the default branch. Currently you could do `on git.push where event.branch is "main"`, but something like "event.branch is event.default_branch" would be even better. I'll work on adding that. The value add currently is that Cicada is FOSS, platform agnostic (works with GitHub/Gitlab), and uses a language that consolidates the workflows and scripts into one manageable file format. Existing CI systems are already packed with features that people expect, so the current struggle is catching up to this and then adding more on top of that. I'm trying to focus on what sets Cicada apart: That it gives you more control over your workflows, while being expressive and easier to manage then YAML and shell scripts. [1]: https://github.com/Cicada-Software/cicada/commit/2659f79b5009a7a53ae28b2c465f956cbfd81127 https://github.com/Cicada-Software/cicada/commit/2659f79b500...

spankalee 3y ago

This is very light on details. Does it actually allow you to run your GitHub Actions on your own servers?

capableweb 3y ago
This is the syntax of the pipeline it seems: https://github.com/Cicada-Software/demo/blob/main/hello_world.ci https://github.com/Cicada-Software/demo/blob/main/hello_worl... Short answer: No
- deleted 3y ago
  [deleted]
dboreham 3y ago
See comment above for how you can do that.

barkingcat 3y ago

There is a collision in names: Cicada, this CI tool, uses a DSL (domain specific language) to write configuration, and this DSL is referred to as "Cicada language", and blasted in marketing copy as a "real programming language" on https://cicada.sh/ https://cicada.sh/ . The Cicada DSL is documented in https://cicada.sh/docs/ci-lang/index.html https://cicada.sh/docs/ci-lang/index.html However, this is a completely different language from Cicada language, a programming language and theorem prover hosted at https://cicada-lang.org/ https://cicada-lang.org/ and https://github.com/cicada-lang/cicada https://github.com/cicada-lang/cicada . This name collision is very confusing, and I wonder why Cicada the CI tool didn't just stick to python, since it is also a "real programming language."

deleted 3y ago
[deleted]
bogwog 3y ago
I didn't know about that other Cicada language, but the part about using a DSL for this does seem unnecessary to me. I haven't dived into the docs at all, so if anyone has I'd be curious to know what this DSL offers over Python, Starlark, or any other existing programming language? Especially since, as far as I can tell, this is built on Python. So you already have a working Python installation at least.
- dosisod 3y ago
  Creator of Cicada here. I created a new DSL because I wanted a more declarative, expressive, yet capable file format, something like Dockerfile but with programming capabilities. In terms of "just running commands", Python (or any other language) is just as good if not better. Where Cicada shines though is that it makes higher level concepts like conditional execution and caching front end center, so you can describe more with less lines. The DSL itself isn't very mature at the moment, but I plan on adding stuff like allow/deny capabilities and more in the future.

dbingham 3y ago

This is a misleading title. Being yet another CI tool is not the same thing as being a cross platform version of Github Actions and Gitlab CI. A cross platform version of Github Actions would allow you to run your actions using your own tooling.

dboreham 3y ago
Also this exists: https://github.com/nektos/act https://github.com/nektos/act which is conveniently packaged in a GH-a-like system by: https://gitea.com/gitea/act_runner https://gitea.com/gitea/act_runner in https://github.com/go-gitea https://github.com/go-gitea
- tao_at_garden 3y ago
  And there's us, https://garden.io https://garden.io, that does totally portable CI pipelines with caching from builds to tests: https://github.com/garden-io/garden https://github.com/garden-io/garden I made an irreverent short on why your CI pipelines ought to be portable and runnable locally. Number one reason? Give your developers their time (and sanity) back. Pushing to Git in the inner loop disrupts flow and shatters attention. https://www.youtube.com/watch?v=JFnan6s2cDg https://www.youtube.com/watch?v=JFnan6s2cDg
  - dosisod 3y ago
    Nice product! I haven't heard of Garden until now. Adding a CLI for Cicada has been on my todo list for a while, and is long overdue! This should be easy enough to do.
  - eysi 3y ago
    Another Gardener here :) Just to add to what Tao was saying, our pipelines are not only portable but also "smart". Instead of having to specify every step of a job—either in code or config—you instead tell Garden that it's e.g. a build of type container, or a deploy of type Helm, and our plugin system figures out the rest (of course with escape hatches when needed). We also track the files that go "into" each job and cache the results, so the same job never has to run twice if the code doesn't change. So if you have a large distributed system and a change only affects a small part of it, you don't need to re-run everything. It can save _a lot_ of time. Unrelated, but love that the Cicada team created a Treesitter grammar for Neovim!

debarshri 3y ago

Something I have noticed lately, First startup idea that every Devops person starts with is to create a CI/CD project because they think world is inefficient and workflows can be optimized, and then they pivot to other ideas.

j45 3y ago
So would that mean there is still room for improvement?
- debarshri 3y ago
  I think market is very mature for CI/CD, it often boils down to price war. Economics of operating your own CI/CD in org like this one is fairly bad compared to Github actions. Especially, the way Github bundles actions with the other services as a buyer it definitely feels like a better deal than buying best in class CI/CD. Github actions gets the job done. The slow changing nature of pipelines makes then candidate for not being touch once the configuration is set. So, to answer your question, in my opinion - is there a room for improvement - yes, but the value is minuscule for a customer to switch to a different provider in CI/CD is working fairly well.
  - j45 3y ago
    Appreciate that. The economics may appear bad for not using GitHub, but there’s still valid needs for code that isn’t hosted in the cloud. For those cases it is reasonable to see or know what’s out there.
    - debarshri 3y ago
      Fair enough.
intelVISA 3y ago
I always thought its ease of creation (few weekends) was the main driver?
3y ago

leonheld 3y ago

What I really want is a 'Grand Translator' tool that allows any CI pipeline definition to be translated into any other CI pipeline definition. I want to take any .gitlab-ci.yml and magically translate it to a github workflow, and vice-versa. I know it isn't impossible, but it's a heck lot of work to get it right with all the hidden features behind declarative pipelines.

ska 3y ago
If doing this sort of a migration is a real possibility, it's an argument for having the bare minimum logic in the CI tool and as much as you can in your own scripts...
- dindresto 3y ago
  That's actually what we did for our monorepo. We had a huge Gitlab pipeline of multiple steps and jobs, now it's a single job pipeline that builds, tests and deploys 15 different projects, all powered by Nix and some Python scripts.
- leonheld 3y ago
  Ugh, yes, but then I fall into the "too complex for bash, too simple for Python" trap. I feel like everything goes to hell the moment someone writes a Python class to deal with /something/ in a build pipeline. For bash, well, we all know where that goes... the other options are simply too obscure for me to invest in. But I agree with your assessment, as much as the classic script: - chmod +x pipeline.sh - ./pipeline.sh makes me want to die.
sakopov 3y ago
We need a "terraform" for build pipelines.
- _joel 3y ago
  oh god, please, no. :) I mean, go for it if you want but I'm not sure why you'd need to maintain so many heterogeous pipelines that would warrant a tool like this.

endigma 3y ago

You should change your name. "Cicada" was also a CI/CD tool recently sunset by AWS after acquiring Fig. I maintain the community fork so I happen to know Amazon still holds a trademark on "Cicada"[1]. We rebranded to "Katoa" but you're going to run into them directly especially if you have any commercial plans. 1: Can't link, you can search "Cicada" and "Amazon" on the USPTO TESS

joshuaissac 3y ago
Here it is: https://tsdr.uspto.gov/search.action?sn=98150978 https://tsdr.uspto.gov/search.action?sn=98150978 > online non-downloadable software for operating and managing continuous integration and continuous delivery pipelines in computer programming environments
deleted 3y ago
[deleted]

biugbkifcjk 3y ago

I'm going to plug https://onedev.io/ https://onedev.io/ its awesome. Its self hosted and has its own tooling for CI/CD. I feel it doesn't get enough love, but I've been using it for years for my own stuff.

trescenzi 3y ago
Oh this looks really neat. I've been slowly hand rolling a home server. Started with Gogs felt that was too heavy, now I'm at cgit + gitolite which is awesome but I'm not sure what I want to do when it comes to CI/CD. This is probably on the scale of Gogs but something about it feels more reasonable lol.
- biugbkifcjk 3y ago
  There is a bit of a learning curve, but there is for any new platform. The devs are very responsive and looks like they're doing an enterprise version for funding. It uses Agents which run the tasks in the CI/CD jobs.
deleted 3y ago
[deleted]

moffkalast 3y ago

Cicada 3301 finally produced something? /s

thepostman0 3y ago
Unsigned though

reactordev 3y ago

Mods, can we change the title to reflect what this really is, Open Source CI platform (not GitHub Actions, not GitLab CI). Unless I’m mistaken, this project doesn’t use their flavors of yaml. So calling it a “version of” is a lie.

deleted 3y ago
[deleted]

xinayder 3y ago

What's the difference between this and Gitea/Forgejo Runners that are compatible with GitHub Actions?

djha-skin 3y ago

I have logic that needs to be put into the build. I can put it into a DSL or I can put it into a bash script. A bash script can be run on a developer's machine. This a bash script allows the developer to test the build on their own machine without the need for running an entire build through the CI system first. I therefore don't see the need for all that DSL stuff that's designed around not needing a bash script. I still need the batch script for testability purposes. I'm fine with the DSL otherwise; switching from Jenkins to GitHub actions isn't a big deal especially when all you're going to do is run a bash script.

throwawaaarrgh 3y ago

This looks really interesting, but I wish people would instead contribute to WoodpeckerCI. It's a FOSS fork of Drone, and Drone really is the pinnacle of CI for me. Only what you need, no fat, simple, functional, modern, easy. IMHO the only thing Woodpecker really needs right now is help finishing their core implementations of K8s support and support for the big 3 DVCS vendors.

skinner927 3y ago
WoodpeckerCI only works with containers. Many of us are not building web apps.

iFire 3y ago

LICENSE AGPL https://github.com/Cicada-Software/cicada/blob/main/LICENSE https://github.com/Cicada-Software/cicada/blob/main/LICENSE I.. wouldn't use to build commercial software.

Yasuraka 3y ago
You seem to have no idea what AGPL entails or be maliciously spreading FUD.
- robertlagrant 3y ago
  Maybe attack the argument, not the person?

thamer 3y ago

If anyone from the project is reading this, the multiple misspellings or typos featured prominently on your website make it look very amateurish (imo). Between "You deserve better then YAML", "autotomatically escaped", and other weird phrasing, this doesn't really inspire confidence.