4 ms·

I am so grateful for this. As a non-Java person, using the Groovy pipelines was incredibly painful to incrementally fix on the server. Nothing like pushing 20 c

I am so grateful for this. As a non-Java person, using the Groovy pipelines was incredibly painful to incrementally fix on the server. Nothing like pushing 20 commits in a row and forcing a build, only to find each time that an odd syntax error.

stefan_ 8y ago
Still have to do that to build the thing in the first place. I do not understand at all how every CI solution out there makes it entirely impossible to trivially test a pipeline. Here we are building something whose entire point is that it can run wherever, but the one place it can't run is my local machine? It's like they are doing it out of spite.
- eddieh 8y ago
  I seriously don't understand how anyone can use most of these CI solution. Here is what everyone should demand from a CI solution: * Actual code that specifies a pipeline (not a GUI/config-file/DSL) * Supports any SCM, workflow, build system, and language * Supports complex mixed-language, mixed-build-system projects * Doesn't make assumptions about project's structure and processes * Able to run locally (but scales to clusters, etc) * Written in a common scripting language * Completely end-to-end customizable in the scripting language it's written in The only one I know that fits is BuildBot, but practically nobody uses it. [edit: formatting]
  - scarface74 8y ago
    Well if the whole world has a different perspective than you, have you thought that maybe their wants/needs are different?
    - eddieh 8y ago
      Sure that could be the case, but I think most people that set up and use CI just pick Jenkins or whatever on a whim, then we all read about horror stories and bad experiences because the solution they picked is a square peg and they're forcing it in a round hole.
      - scarface74 8y ago
        Why do I need all of that bespoked complexity, my yml file is just there to tell the build system the scripts or shell commands I want to run - that are also in my git repo. Fundamentally, a CI system for a simple application just has to gather dependencies, build a package/run automated tests and store the artifact somewhere. A deployment system just has to copy the deployment package to a server/group of servers, and do some type of installation process based on an automatic (Dev/integration environment) or manual approval process (who ever is responsible for pushing to any other environment has to approve the release). There are a million ways to skin the cat but if your build process or deployment process is too complex, maybe it’s more of a question of the maturity of your framework or tooling. My build process for .Net web apps is: - nuget restore - msbuild - run nunit - zip artifacts based on version. My deployment process is - run a CloudFormation file to configure AWS resources - deploy code to a VM - run one or two commands to configure IIS or install a Windows service. - run a few AWS CLI commands to start autoscaling, reconfigure API gateway, etc. No build servers to maintain, just an agent running on the target VM for deployment and a slightly custom Docker image for builds. —- For Lambda functions and scripting languages it’s even simpler... Build: - import deprndencies - create zip file Deployment - run CloudFormation YML file to deploy lambda The CF file can be the deployment step. If I need some programmatic, I can create a custom lambda backed resource that is run when the CF file is run.
- singingfish 8y ago
  Hah. I have a short perl script that trivially tests individual components of gitlab pipelines (parses the yaml, extracts the script components, saves them to a temp file then executes the temp file as bash script). The code is short but a little contorted. Works well though. Will be releasing into the wild in a couple of months.
dunckr 8y ago
I had the same problem. Luckily jenkins provides an API which you can validate your jenkinsfile. I setup the deploy to always lint before attempting to run. https://jenkins.io/doc/book/pipeline/development/#linter https://jenkins.io/doc/book/pipeline/development/#linter
- sciurus 8y ago
  That only works for declarative pipelines, not scripted ones, though.
  - slavik81 8y ago
    That's another problem with the existing pipeline. There are so many slightly different ways of setting them up. They're all similar enough to make tutorials and documentation confusing, but different enough that that it's a problem when you mix them up. All I want to do is poll an SVN repository for changes on any branch. If there is a change on a branch, I want to check it out, run the build script, and archive the resulting binary on success or send an email on failure. I spent a few hours trying to figure out how to do it in their Groovy DSLs but I eventually gave up. That was mostly easy with the old Jenkins script jobs (though I had to copy it for each branch), and I find Travis CI to be a bit limited but very straightforward, so I was kinda disappointed. I'm happy to see they're working in making it easier to use Jenkinsfiles. There's potential there but the UX needs improvement. This might be it.

stefan_ 8y ago

Still have to do that to build the thing in the first place. I do not understand at all how every CI solution out there makes it entirely impossible to trivially test a pipeline. Here we are building something whose entire point is that it can run wherever, but the one place it can't run is my local machine? It's like they are doing it out of spite.

eddieh 8y ago
I seriously don't understand how anyone can use most of these CI solution. Here is what everyone should demand from a CI solution: * Actual code that specifies a pipeline (not a GUI/config-file/DSL) * Supports any SCM, workflow, build system, and language * Supports complex mixed-language, mixed-build-system projects * Doesn't make assumptions about project's structure and processes * Able to run locally (but scales to clusters, etc) * Written in a common scripting language * Completely end-to-end customizable in the scripting language it's written in The only one I know that fits is BuildBot, but practically nobody uses it. [edit: formatting]
- scarface74 8y ago
  Well if the whole world has a different perspective than you, have you thought that maybe their wants/needs are different?
  - eddieh 8y ago
    Sure that could be the case, but I think most people that set up and use CI just pick Jenkins or whatever on a whim, then we all read about horror stories and bad experiences because the solution they picked is a square peg and they're forcing it in a round hole.
    - scarface74 8y ago
      Why do I need all of that bespoked complexity, my yml file is just there to tell the build system the scripts or shell commands I want to run - that are also in my git repo. Fundamentally, a CI system for a simple application just has to gather dependencies, build a package/run automated tests and store the artifact somewhere. A deployment system just has to copy the deployment package to a server/group of servers, and do some type of installation process based on an automatic (Dev/integration environment) or manual approval process (who ever is responsible for pushing to any other environment has to approve the release). There are a million ways to skin the cat but if your build process or deployment process is too complex, maybe it’s more of a question of the maturity of your framework or tooling. My build process for .Net web apps is: - nuget restore - msbuild - run nunit - zip artifacts based on version. My deployment process is - run a CloudFormation file to configure AWS resources - deploy code to a VM - run one or two commands to configure IIS or install a Windows service. - run a few AWS CLI commands to start autoscaling, reconfigure API gateway, etc. No build servers to maintain, just an agent running on the target VM for deployment and a slightly custom Docker image for builds. —- For Lambda functions and scripting languages it’s even simpler... Build: - import deprndencies - create zip file Deployment - run CloudFormation YML file to deploy lambda The CF file can be the deployment step. If I need some programmatic, I can create a custom lambda backed resource that is run when the CF file is run.
singingfish 8y ago
Hah. I have a short perl script that trivially tests individual components of gitlab pipelines (parses the yaml, extracts the script components, saves them to a temp file then executes the temp file as bash script). The code is short but a little contorted. Works well though. Will be releasing into the wild in a couple of months.

dunckr 8y ago

I had the same problem. Luckily jenkins provides an API which you can validate your jenkinsfile. I setup the deploy to always lint before attempting to run. https://jenkins.io/doc/book/pipeline/development/#linter https://jenkins.io/doc/book/pipeline/development/#linter

sciurus 8y ago
That only works for declarative pipelines, not scripted ones, though.
- slavik81 8y ago
  That's another problem with the existing pipeline. There are so many slightly different ways of setting them up. They're all similar enough to make tutorials and documentation confusing, but different enough that that it's a problem when you mix them up. All I want to do is poll an SVN repository for changes on any branch. If there is a change on a branch, I want to check it out, run the build script, and archive the resulting binary on success or send an email on failure. I spent a few hours trying to figure out how to do it in their Groovy DSLs but I eventually gave up. That was mostly easy with the old Jenkins script jobs (though I had to copy it for each branch), and I find Travis CI to be a bit limited but very straightforward, so I was kinda disappointed. I'm happy to see they're working in making it easier to use Jenkinsfiles. There's potential there but the UX needs improvement. This might be it.