6 ms·

Thanks for the warning, are there any good tools that integrate with pull request checks that can be self hosted? We self host nearly everything at work except

Thanks for the warning, are there any good tools that integrate with pull request checks that can be self hosted? We self host nearly everything at work except for github. (And it's VERY difficult for me to get anything that costs money approved regardless of price)

prophetjohn 7y ago
It costs money, but for other readers, I've been using Buildkite[0] at two different companies across ~4 years now and it's great. It's _mostly_ self-hosted in that jobs run on your own hardware and Buildkite itself is only the job queue that your self-hosted agents pull work from. They also provide an elastic CI stack[1] so you can have auto-scaling workers running on AWS You still own enough of it that if the service is down, you can just execute the scripts manually to build, test and deploy [0] https://buildkite.com https://buildkite.com [1] https://github.com/buildkite/elastic-ci-stack-for-aws https://github.com/buildkite/elastic-ci-stack-for-aws
wilsonrocks 7y ago
Gitlab let's you do an awful lot and can be self hosted.
bravura 7y ago
And it's VERY difficult for me to get anything that costs money approved regardless of price For me, this is a big red flag.
- penagwin 7y ago
  I should probably emphasize that it's difficult for me to get new things approved, but if we NEED it then money is not a problem. The response I get is we don't _NEED_ a CI/CD (and haven't had one for years), so they would rather I find a solution that can run on our hardware for free. I'll add that I enjoy my work environment and they treat me very well, I'm young and I'm known to "like shiny new things". I also have to work on how I pitch why we need certain things, again I'm pretty young so I'm not well practiced in that just yet.
  - jrochkind1 7y ago
    > The response I get is we don't _NEED_ a CI/CD You are not making this sound like less of a red flag! But yeah, if you are both new and young there, you can't just push hard for all the things you want. But in this case... you're right, heh. If I were you, I'd be keeping my eyes out for job opportunities in environments where you can actually learn something about good modern software engineering practices form their example. (Easier said then done, don't I know it).
  - wasyl 7y ago
    For a free, self-hosted CI I strongly recommend TeamCity -- if you can get away with 3 build agents and 100 configurations that come with the free version. I was in your boots some time ago, managing CI before I convinced the company that paying for a hosted CI is cheaper than me maintaining our self-hosted whatever CI we were trying at that point :) Jenkins was atrocious and I spent ungodly amount of time dealing with all kinds of issues, while TeamCity was mostly configure-and-forget. Initial setup is pretty straightforward, so is later migration to a real database for example. Updates are seamless and we had some projects on it even after we started using paid CI (Bitrise), because there was no motivation to migrate as everything was running smoothly.
disconnected 7y ago
> Thanks for the warning, are there any good tools that integrate with pull request checks that can be self hosted? Recently at work we've done an analysis of a of CI/CD tools. Many of them are self-hosted, free (and open source) and support a variety of workflows - including pull request related checks (either by polling of via webhooks). A cursory search will yield you a lot to play with, so... go ahead and do that. That said, if you don't want to think about it too much, you can't go wrong with Jenkins. Some people will suggest GitLab. I'd steer clear of GitLab, though, because of their operational incompetence [1] and their funny ideas about mandatory corporate espionage [2] - I mean, telemetry - which they only backed away from because people yelled at them. That second one was enough to disqualify them in our analysis (the first one just cemented the idea). [1] https://about.gitlab.com/blog/2017/02/10/postmortem-of-database-outage-of-january-31/ https://about.gitlab.com/blog/2017/02/10/postmortem-of-datab... [2] https://www.theregister.co.uk/2019/10/30/gitlab_backtracks_on_plan_to_add_usage_telemetry_after_user_protests/ https://www.theregister.co.uk/2019/10/30/gitlab_backtracks_o...
- 0xEFF 7y ago
  Gitlab CI is great if you need only one pipeline to execute per branch. As soon as you need multiple pipelines per branch it doesn’t work well. Most small projects need only one pipeline and are well suited. Other things like terraform needing multiple pipelines for multiple environments are better suited for a CI platform that handles multiple pipelines.
- mnutt 7y ago
  It seems like their gitlab.com uptime isn't that relevant when the question was about self-hosted CI. And while I don't like the idea of telemetry in self-hosted products either, calling it "corporate espionage" is hyperbole and I don't know how much more one could expect from their response other than apologizing and backtracking.
xtracto 7y ago
I echo the Gitlab recommendation. Setting up CI/CD pipelines is beautifull, easy and all code-backed. And you can self hist it. Really. Having one repo checked out into a docker container with a specific node version toolchain and running your test suite with code coverage + publishing html reporting is a matter of a .gitlab_ci.yml config file with 6 to 10 lines .
ethomson 7y ago
Product Manager for GitHub Actions here. You can self-host the GitHub Actions runners. This means that you can run CI/CD workflows on your own hardware, while your source code is hosted on github.com itself. https://help.github.com/en/actions/automating-your-workflow-with-github-actions/about-self-hosted-runners https://help.github.com/en/actions/automating-your-workflow-...
- petrikapu 7y ago
  Can you give some insight when shared self-hosted for private repos are available? It was super easy to setup for single project. I’d like to share beefy AWS instance for few more private repos. Keep on good work!
- penagwin 7y ago
  We're currently trying to use Github Actions but we've run into a few issues. The first is that when our team members push pull requests from their forked branches it doesn't trigger checks on the pull request. I understand this is for security reasons, but in our case the repository is private and all forks are considered trusted. The other is that there's currently no way to label specific workers. Because of how our network/firewall is segmented we would like to be able to specify a worker for Staging/Dev/Production separately. The closest thing I found was this [0] pull request from October, but it doesn't add the ability to add custom labels. Is there any timeline for either of these features (especially the labels)? [0] https://github.com/actions/runner/pull/130 https://github.com/actions/runner/pull/130

prophetjohn 7y ago

It costs money, but for other readers, I've been using Buildkite[0] at two different companies across ~4 years now and it's great. It's _mostly_ self-hosted in that jobs run on your own hardware and Buildkite itself is only the job queue that your self-hosted agents pull work from. They also provide an elastic CI stack[1] so you can have auto-scaling workers running on AWS You still own enough of it that if the service is down, you can just execute the scripts manually to build, test and deploy [0] https://buildkite.com https://buildkite.com [1] https://github.com/buildkite/elastic-ci-stack-for-aws https://github.com/buildkite/elastic-ci-stack-for-aws

wilsonrocks 7y ago

Gitlab let's you do an awful lot and can be self hosted.

bravura 7y ago

And it's VERY difficult for me to get anything that costs money approved regardless of price For me, this is a big red flag.

penagwin 7y ago
I should probably emphasize that it's difficult for me to get new things approved, but if we NEED it then money is not a problem. The response I get is we don't _NEED_ a CI/CD (and haven't had one for years), so they would rather I find a solution that can run on our hardware for free. I'll add that I enjoy my work environment and they treat me very well, I'm young and I'm known to "like shiny new things". I also have to work on how I pitch why we need certain things, again I'm pretty young so I'm not well practiced in that just yet.
- jrochkind1 7y ago
  > The response I get is we don't _NEED_ a CI/CD You are not making this sound like less of a red flag! But yeah, if you are both new and young there, you can't just push hard for all the things you want. But in this case... you're right, heh. If I were you, I'd be keeping my eyes out for job opportunities in environments where you can actually learn something about good modern software engineering practices form their example. (Easier said then done, don't I know it).
- wasyl 7y ago
  For a free, self-hosted CI I strongly recommend TeamCity -- if you can get away with 3 build agents and 100 configurations that come with the free version. I was in your boots some time ago, managing CI before I convinced the company that paying for a hosted CI is cheaper than me maintaining our self-hosted whatever CI we were trying at that point :) Jenkins was atrocious and I spent ungodly amount of time dealing with all kinds of issues, while TeamCity was mostly configure-and-forget. Initial setup is pretty straightforward, so is later migration to a real database for example. Updates are seamless and we had some projects on it even after we started using paid CI (Bitrise), because there was no motivation to migrate as everything was running smoothly.

disconnected 7y ago

> Thanks for the warning, are there any good tools that integrate with pull request checks that can be self hosted? Recently at work we've done an analysis of a of CI/CD tools. Many of them are self-hosted, free (and open source) and support a variety of workflows - including pull request related checks (either by polling of via webhooks). A cursory search will yield you a lot to play with, so... go ahead and do that. That said, if you don't want to think about it too much, you can't go wrong with Jenkins. Some people will suggest GitLab. I'd steer clear of GitLab, though, because of their operational incompetence [1] and their funny ideas about mandatory corporate espionage [2] - I mean, telemetry - which they only backed away from because people yelled at them. That second one was enough to disqualify them in our analysis (the first one just cemented the idea). [1] https://about.gitlab.com/blog/2017/02/10/postmortem-of-database-outage-of-january-31/ https://about.gitlab.com/blog/2017/02/10/postmortem-of-datab... [2] https://www.theregister.co.uk/2019/10/30/gitlab_backtracks_on_plan_to_add_usage_telemetry_after_user_protests/ https://www.theregister.co.uk/2019/10/30/gitlab_backtracks_o...

0xEFF 7y ago
Gitlab CI is great if you need only one pipeline to execute per branch. As soon as you need multiple pipelines per branch it doesn’t work well. Most small projects need only one pipeline and are well suited. Other things like terraform needing multiple pipelines for multiple environments are better suited for a CI platform that handles multiple pipelines.
mnutt 7y ago
It seems like their gitlab.com uptime isn't that relevant when the question was about self-hosted CI. And while I don't like the idea of telemetry in self-hosted products either, calling it "corporate espionage" is hyperbole and I don't know how much more one could expect from their response other than apologizing and backtracking.

xtracto 7y ago

I echo the Gitlab recommendation. Setting up CI/CD pipelines is beautifull, easy and all code-backed. And you can self hist it. Really. Having one repo checked out into a docker container with a specific node version toolchain and running your test suite with code coverage + publishing html reporting is a matter of a .gitlab_ci.yml config file with 6 to 10 lines .

ethomson 7y ago

Product Manager for GitHub Actions here. You can self-host the GitHub Actions runners. This means that you can run CI/CD workflows on your own hardware, while your source code is hosted on github.com itself. https://help.github.com/en/actions/automating-your-workflow-with-github-actions/about-self-hosted-runners https://help.github.com/en/actions/automating-your-workflow-...

petrikapu 7y ago
Can you give some insight when shared self-hosted for private repos are available? It was super easy to setup for single project. I’d like to share beefy AWS instance for few more private repos. Keep on good work!
penagwin 7y ago
We're currently trying to use Github Actions but we've run into a few issues. The first is that when our team members push pull requests from their forked branches it doesn't trigger checks on the pull request. I understand this is for security reasons, but in our case the repository is private and all forks are considered trusted. The other is that there's currently no way to label specific workers. Because of how our network/firewall is segmented we would like to be able to specify a worker for Staging/Dev/Production separately. The closest thing I found was this [0] pull request from October, but it doesn't add the ability to add custom labels. Is there any timeline for either of these features (especially the labels)? [0] https://github.com/actions/runner/pull/130 https://github.com/actions/runner/pull/130