5 ms·

Hey Jeff, This is hitting me hard, too. I hope this is the right moment for me to pipe up and say this: I always just do the minimum with these CI integration

• steveklabnik 6y ago
  I don’t think either of them really “benefit from the advanced features” any more than any other language. Also, both have builds that are generally more cross platform than a shell script, given that they’re native on all platforms, and shell is not.
• monkeybutton 6y ago
  >do the minimum with these CI integrations to get my shell script to run This. 100%. Where I'm working is migrating from Bamboo to Jenkins. For my team the new process is just calling "./build-test $params" while others are stuck re-implementing various stages, tasks and blah blah blah. Also an added bonus to this is having the full history of changes to the CI/CD process tracked in git.
  • redis_mlc 6y ago
    I call mine make.sh. Now get off my lawn!
    • monkeybutton 6y ago
      Hey! I started with C projects and make scripts and I long for the simplicity of those days! Obtuse web front ends, bunches of build server pools segregated by random capabilities, job queues clogged by competing developer teams and on and on is no fun.
    • saurik 6y ago
      I wouldn't want something called "make.sh" to install tooling on the computer, though, and that is an important step of CI stuff.
      • aprdm 6y ago
        Run it in a docker container so that it does the same in your computer and in the CI worker. make prepare make build make test make release Working the same waylocally and in jenkins makes things very easy. What it does is up to the team but make prepare usually pulls some docker image and install dependencies, make build builds something inside the docker image, make test runs test inside it and make release publishes the docker img
        • laumars 6y ago
          Or use a CI/CD tool that natively supports Docker, like Concourse. Added bonus is that pipelines are defined in YAML rather than web GUI so you can version control the pipelines themselves and not just any build or test suites.
          • aprdm 6y ago
            Jenkins pipelines are written in Groovy and you do not need to use the web ui, everything is exposed via its API (i.e: new job creation). I have heard good things about Concourse, want to give it a try in the future!
            • laumars 6y ago
              > Jenkins pipelines are written in Groovy and you do not need to use the web ui, everything is exposed via its API (i.e: new job creation). There's several ways you can codify Jenkins pipelines, Groovy being just one of them. But ultimately it's secondary to the main design of Jenkins. I'm not taking anything away from Jenkins as a solution - it has been invaluable over the years. But the way we write, test and deploy software has changed since Jenkins rose in popularity and as such we need to rely on different workflows that utilise different tooling. While Jenkins can be used in that way, I'd sooner use something which was primarily designed to be configured via code and used docker by default rather than something that requires enforcement to follow those best practices. That doesn't necessarily mean Concourse, there's Travis, Circle CI, AWS CodeBuild and a bunch of others that all default to that kind of workflow too, but it does mean I'm unlikely to ever advocate Jenkins again in future jobs.
  • est31 6y ago
    > while others are stuck re-implementing various stages, tasks and blah blah blah. Also an added bonus to this is having the full history of changes to the CI/CD process tracked in git. Can't those stages, tasks, etc. be specified in the CI specific config files of the repository? Specifying them via an UI seems horrible to me. At least Github actions is all file-specified (except for secrets).
    • manigandham 6y ago
      It still requires conversion of config files and formats vs a script that is the same everywhere.
      • est31 6y ago
        Yes, that's a valid point, and definitely a disadvantage. I was referring to the point that it's not tracked in git.
  • InsaneOstrich 6y ago
    Are you moving to Jenkins because of the recent Atlassian license changes?
    • monkeybutton 6y ago
      Not entirely but I think it was the straw that broke the camel's back. Apparently our journey to the cloud will have to wait.
      • InsaneOstrich 6y ago
        I think there will still be a on premise version of Bamboo going forward, but all the license changes have made my company consider the possibility of moving from Bamboo to something else. That's kind of a shame, because Bamboo works really well for us.
  • merb 6y ago
    btw. even gitlab does that with their autodevops, besides that they create gitlab-runner: https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Build.gitlab-ci.yml#L17 https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitla... https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/DAST-Default-Branch-Deploy.gitlab-ci.yml#L8 https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitla... https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/blob/master/src/bin/auto-deploy https://gitlab.com/gitlab-org/cluster-integration/auto-deplo... I found these fascinating, I mean they still have lots of things in their ci file, but basically the have tons of shell scripts that would work without their gitlab-runner. Not affiliated with gitlab I just tried to figure out how some of their stuff works to have the same k8s integration than their autodevops offers.
• geerlingguy 6y ago
  Makefiles are the gold standard there, warts and all. But in every case, unless you encode a whole plethora of hacks in your golden build script, you still need minimal environment setup, like Python + Docker + [insert library here]. Most of my Travis files are only 10-20 lines long, but I work in a lot of different types of environments, so it's not always as easy as "three lines of bash and a docker run command". Plus I have a number of legacy projects which did use some of Travis' more exotic features, and those will take longer. Not just for myself, but I'm thinking of tons of smaller OSS libraries and projects that are minimally maintained yet have a large footprint under more popular and maintained software!
  • sounds 6y ago
    I'd be interested to have a deeper discussion with you about Makefiles, sometime. We've actually migrated all our projects off Makefiles. Developers hate having to live within the constraints.
• curryhoward 6y ago
  Small plug: consider using Toast [1] to define your CI jobs. It's essentially Make + Docker. You define whatever targets you want (build, test, lint, etc.) and the dependencies between them in a YAML file, and Toast runs them in Docker containers. You can run your jobs on any machine that has Docker—including CI systems like Travis and GitHub Actions. You aren't locked into any CI system, and you can also run your jobs locally. [1] https://github.com/stepchowfun/toast https://github.com/stepchowfun/toast
• qppo 6y ago
  Once you start caring about cross-platform, incremental, distributed builds with a bunch of people committing to a monorepo those advanced features start becoming attractive. Shell scripts handle the 'run this build/test task' but not 'pull these artifacts from another set of tasks, run this task to produce a bunch more artifacts, cache them for later.' In these scenarios the lines between build system, package manager, and task runner all become pretty blurred. Haven't found an all in one solution that I would call "portable" yet.
  • emptysea 6y ago
    Sounds like your describing what I like to call a fancy build system Something like Bazel[0], Buck[1] Pants[2], etc. One of their main features is robust caching of build artifacts, which is helpful for larger projects. But they require some major investment compared to your typical `scripts/build.sh`. [0]: https://bazel.build https://bazel.build [1]: https://buck.build https://buck.build [2]: https://www.pantsbuild.org https://www.pantsbuild.org
    • qppo 6y ago
      I have a lot of ideas (or rather, one idea) about how to fix my CI problems and "fancy build system" is pretty apt. Bazel is the closest thing I've seen to what I want.
    • brabel 6y ago
      Gradle can also do everything imaginable, we've managed to get rid of basically all our horrible bash scripts and wrote most build stuff using Gradle API and using a proper programming language (put actual code in Gradle's buildSrc/, only configure it in build.gradle, and you've got a extremely powerful build).
      • moonbug 6y ago
        sweet summer child.
      • hinkley 6y ago
        Shebangs are also your friend, if your primary language can be used that way. The cons of that solution are that there’s a tendency to make them ornate, and the trap of circular dependencies. Reusing an internal library in your build scripts, you have it building itself which can get out of hand quickly.
• DrBazza 6y ago
  > I always just do the minimum with these CI integrations to get my shell script to run. Absolutely. Moving CI systems always shows up where your build logic has leaked into the CI system, rather than staying in your repo. If you can't identically build your code from the command line and from the CI system, you're going to have problems.
• hinkley 6y ago
  You want coworkers and collaborators to have such an easy time validating build failures locally that only an idiot or an asshole won’t do anything about a problem they may have caused. Developers hate being thought of as idiots, and being an asshole is a problem you can take to HR. So for the people who don’t pitch in on general principle, straightforward processes help bring them around. And in general that means not leaning on clever CI features, and instead leaning hard on the CLI tools your ecosystem supports. There are exceptions to every rule of course. Features around scheduling have little to no impact on what gets built and how, just when and why. Similarly, analytics have second order effects on how and when, but once a problem is discovered you can mostly ignore them for a while. Integration with external tools and services on the other hand, should always be questioned. Do I have another way to do this mostly outside of the CI system?
• __s 6y ago
  My .travis.yml for a node project is pretty simple: https://github.com/serprex/openEtG/blob/542c645c53dba36d665342975b566487a27b62a8/.travis.yml https://github.com/serprex/openEtG/blob/542c645c53dba36d6653... Python's easy too: https://github.com/serprex/aespython/blob/c3256b656308af8119eedb69b0ac2aa85040f38d/.travis.yml https://github.com/serprex/aespython/blob/c3256b656308af8119... A Rust repo I contribute to has a .travis.yml not created by me, but still pretty simple: https://github.com/dpc/hex2d-rs/blob/master/.travis.yml https://github.com/dpc/hex2d-rs/blob/master/.travis.yml edit: updated node project to Github Actions https://github.com/serprex/openEtG/commit/4196992bb23ce43f3460de960cc0bc05a45ed5bb https://github.com/serprex/openEtG/commit/4196992bb23ce43f34...