8 ms·
How to set up continuous deployment using free hosted tools
- stephengillie 9y agoDid Travis replace Jenkins? Thanks for posting this. I'm looking to get any application and start deploying through a CI/CD process, to learn how this process works. Any application suggestions for a CI "Hello World"?
- oneweekwonder 9y agoHave a look at gitlab ci[0], the quick start was easy. Got it working online and self-hosted. https://about.gitlab.com/features/gitlab-ci-cd/ https://about.gitlab.com/features/gitlab-ci-cd/
- aaronbrager 9y agoI think you mean CD “Hello World”, not CI. You can make a “hello world” web app (or even just an API endpoint) in your favorite language and set it up to deploy on each commit. CD means each commit that passes tests is deployed CI means your code is regularly integrated (merged) vs having long QA / code review cycles. CI and CD work great together but you can have one without the other.
- stephengillie 9y agoYes, I want to better understand the process of committing code to the repository of an application, have it run automated tests, report via a communications tool the result, and automatically deploy if successful. Understanding this efficient process is high in demand and thus highly profitable. I've created and worked on several projects that have involved APIs, reading from Zendesk, and writing to Slack or HTML, among other places. I've supported Jenkins and related tools multiple times in production. Now I want to fill this gap in my knowledge.
- korzun 9y agoYou should use Jenkins instead. There is no guarantee that Travis will remain free or remain at all.
- lmm 9y agoTravis is a lot more user-friendly than Jenkins and they run it for you, whereas Jenkins you usually self-host. (Jenkins is a lot more flexible and has a lot going for it, but Travis is much easier to get up and running)
- geofft 9y agoTravis is a third-party service. Jenkins is (usually) software you run yourself. Travis will run the infrastructure for you and will not charge you if your project is public on GitHub. So that makes it an extremely popular choice for open-source projects: sign into Travis with your GitHub credentials, click a checkbox on your repo, and add and commit a .travis.yml file, and now you have CI that is well-integrated with GitHub and you don't have to run any infrastructure or pay for any server hosting. Of course, you have much less control over Travis than Jenkins. Two cases where Jenkins remains obviously better are when you need custom environments that aren't easily portable to Travis (Windows support is a big one, although there is a company called AppVeyor that is something like a Travis for Windows; licensed software is another big one), or if you have strong security/confidentiality requirements (e.g., you're in a regulated industry) and using random cloud services isn't a good option.
- drinchev 9y agoOnly comment I would make is that you can use CircleCI instead of Travis to get for free running tests on private repos. The configuration is similar and the benefits as well.
- simonw 9y agoThanks for the tip - I just added a note about Circle CI to the article.
- oneweekwonder 9y ago> Need more? Pay for it! or be willing to move vendor and re-implement the stuff. For comparison my magic combination(and not all free or realistic for everyone): - gitlab - jenkins/looking into gl ci - linode/cloudafrica(ah-hah hosting is not free) - sentry is a tough one. We rolling a custom mqttwarn/zabbix solution - irc(fine I will take your slack and use the irc-bridge pfftt)
- aloisdg 9y agoirc is fine but if you want an alternative, you can look into riot/matrix (irc bridge), mattermost, rocket chat, etc. For hosting, I am looking to host my own server for fun and learning.
- jherdman 9y agoI really wanted to use Gitlab for my next side project, but Circle CI doesn't support them, and I really don't want to pay Travis for my shitty proj. Hopefully this changes some day soon.
- deleted 9y ago[deleted]
- enzanki_ars 9y agoGitlab has great built in support for CI [0], and you can easily add and run your own CI runners [1]. [0]: https://docs.gitlab.com/ce/ci/README.html https://docs.gitlab.com/ce/ci/README.html [1]: https://docs.gitlab.com/ce/ci/runners/README.html https://docs.gitlab.com/ce/ci/runners/README.html
- markbnj 9y agoWe use gitlab's built-in runners and pipelines for continuous deployment to kubernetes and it works very well. We self-host our repos but the same tools are available through the hosted version.
- geofft 9y agoGitLab's built-in CI is very good, and simpler than Travis for some things. You also have the option (unlike Travis) of registering your own runners in your project settings, and you can do this either on gitlab.com or another deployment, without needing to be an admin of the deployment - you can get a key for it in your own project's settings. I'm using this at work, where we have an internal GitLab that I'm not an admin of (and where the GitLab admins probably have no interest in running a CI system), to do a small bit of CI on some things using one of my team's servers as a runner.
- HugoDaniel 9y agoGitHub and Travis are not self-hosted tools. It would be nice to see a setup with something like Gitea[0] and/or Jenkins[1] edit and some other slack alternative like mattermost[2] [0] https://try.gitea.io/ https://try.gitea.io/ [1] https://jenkins.io/ https://jenkins.io/ [2] https://about.mattermost.com/download/ https://about.mattermost.com/download/
- icebraining 9y ago"Free hosted", not "self-hosted".
- growtofill 9y agoThe title says 'hosted', which is the opposite of 'self-hosted'.
- the_mitsuhiko 9y ago> Travis are not self-hosted tools. Pretty sure you can self host travis.
- benhoyt 9y agoI just set up CI tests for some of my Python libraries on GitHub, and another tool to note is Appveyor -- very useful for running tests on Windows (which Travis doesn't do yet). Example: https://github.com/benhoyt/scandir/blob/master/appveyor.yml https://github.com/benhoyt/scandir/blob/master/appveyor.yml and https://ci.appveyor.com/project/benhoyt/scandir https://ci.appveyor.com/project/benhoyt/scandir
- callumlocke 9y agoHeroku does have a free tier, but it's intended for experiments. It's very limited and wouldn't be acceptable for any serious website, certainly not a 'world class' setup. Unless you're very careful to have one free dyno per account, and avoid using features like staging apps, they will power down your server for several hours a day.
- ainiriand 9y agoYou probably should not use the free tier for any serious website or 'world class' setup...
- simonw 9y agoYeah, the “world-class” hook was more about the quality of these tools than their applicability for use at scale for free - I encourage people to pay real money at various points in the article. I’m currently running my blog on a single $7/month Heroku Dyno (plus $9/month for Heroku PostgreSQL) behind a free Cloudflare account - with the right caching headers I can easily handle huge amounts of read traffic. I was going to mention Cloudflare in this piece but I decided it didn’t fit the theme of continuous deployment as well as the other tools.
- ainiriand 9y agoWhen they say that AI is not going to take over dev jobs, show them this thread and tell them that the time cut down because of automation is going to be spent in something more meaningful, like coding!
- mholmes680 9y agoIn my world, its not world-class if i don't get some SOX compliance around it. Its a cool example though.
- dasil003 9y agoSo you equate "world-class" with "enterprise"? I disagree. Tools that scale down can also be world-class, and in fact that ability to run cheaply is a big part of what allows startups to get off the ground and bootstrap their way to eventually challenge the big boys. Heck, even if they stay small, they can still have world-class tech. Once you get big and purchasing decisions are made far from the front lines I'd argue it gets harder to really have the best tech.
- mholmes680 9y ago>>So you equate "world-class" with "enterprise" Well as part of my current world, sure I do. I have to satisfy the needs of not just dev, but operations, compliance, security, business, the people overseas that i've never met, etc. What I'd really like to see are startups that bootstrap this stuff to think that way prior to growing. My original point was that this was cool for the dev crowd and perfect for the blog use case, but "world"-class seems hyperbolic. But if you are working for other people: people walking around who have strung together various automation stacks are a dime a dozen -- I've literally seen 4 different takes this month. Execution matters, and execution for your audience matters more. >> Tools that scale down can also be world-class We are of the same opinion. I love me some elegant free solutions, and I need to because then I can judge what I'm buying when I need to buy something. >>Once you get big and purchasing decisions are made far from the front lines I'd argue it gets harder to really have the best tech. I guess I see where you could be coming from, but it doesn't HAVE TO be that way does it? Isn't the whole point of work to empower yourself to affect the company decision-making. Failing at doing so is sort of, without offending you, you not doing your job. At least that's how I look at it. [edit: formatting]
- korzun 9y agoThe only 'free' tool in the article is Travis CI. It's important to understand that you are effectively locking yourself in under a promise of a free tier and should plan to make sure you can afford the paid subscription in a couple of months or years. Some companies may also sell your data to third-party affiliates, with, or without your consent or knowledge.
- adnanh 9y agoShameless plug: I wrote an open source incoming webhook server that can run your build & redeployment scripts: https://github.com/adnanh/webhook https://github.com/adnanh/webhook
- nullify88 9y agoThank you very much for this! We use this to listen for Github webhooks and execute jobs with Saltstack.
- milancurcic 9y ago> Once you’ve set up Travis, it will automatically test every commit to every branch Small correction: It will automatically test the latest commit on every push to every branch
- masklinn 9y agoA bit that's missing: bors-ng https://bors.tech https://bors.tech to only merge PRs which passed testing.
- stickfigure 9y agoFor Java/Postgres/React/Google App Engine, here's a skeleton project using CircleCI: https://github.com/stickfigure/gstrap-skeleton/ https://github.com/stickfigure/gstrap-skeleton/ Deployments are git checkout production; git merge master; git push. I love being able to use git log to see what changes are live.
- IgorPartola 9y agoDoes anyone have a solution for a situation where I have a Django backend and a VueJS front end and want to run the Vue build step before deploying? Currently doing this with Git hooks and committing the resultant build, but that is bad for a number of reasons. This is running on Heroku. Would their release phase allow for this without significantly slowing things down? Would it deal with npm installs?
- drdaeman 9y agoWe have this with GitLab CI, although with React and not Vue (but I don't think this matters). We have two stages, `build-assets` and `build-app`, first one runs Webpack etc, and the second one copies the build artifacts into a Docker image with Django and finalizes the image with collectstatic. We're using Docker, but I'm sure the same approach can be used with simple tarballs, OS packages (.debs, etc) or deploy-specific git repo (for Heroku)
- tarr11 9y agoYou do not need to commit your build to git if you are using heroku. If you add the nodejs buildpack to your heroku project, it should detect and build your package.json automatically on heroku. Works with both yarn and npm. You just need to make sure to set NODE_ENV=production in your heroku config. https://devcenter.heroku.com/articles/nodejs-support#build-behavior https://devcenter.heroku.com/articles/nodejs-support#build-b...
- simonw 9y agoYou might be able to solve this using a bin/post_compile script - most standard Heroku buildpacks will run these as part of the build step, and they can be used to run commands that generate files which will then be included in the compiled app that gets deployed to Heroku. Here's an example that might be useful: https://github.com/nigma/heroku-django-cookbook/blob/master/bin/post_compile https://github.com/nigma/heroku-django-cookbook/blob/master/...
- acejam 9y agoI've done this for several apps. We have a large Jenkins 2 cluster and use declarative Jenkinsfile's. For Jenkins, you can accomplish this by simply setting up a Jenkinsfile that has an agent defined in each stage. Build your VueJS app in the first stage with a NodeJS container, and build the second stage with a Python container. Bonus points for building an actual Docker image as your artifact that has Django inside of it with Apache or nginx serving the VueJS bits too.
- analytically 9y agoYou're missing Concourse
- twic 9y agoTwo hours after posting, and this is the only comment about Concourse? jacques_chester must be on holiday. Is it possible get get Concourse running entirely on free tier stuff? I don't think there's a hosted Concourse service yet - airplane.ci is coming, but still in beta - and i imagine you'd struggle to get a whole setup on a t2.micro.
- kellyjandrews 9y agoI'm the developer advocate for Codeship. Codeship (https://codeship.com/features https://codeship.com/features) allows forever free builds for public repos. It has support for both Continuous Integration and Delivery/Deployment. Maybe you can give it a spin as well?
- seer 9y agoI can second codeship - even though its a bit more clunky than Travis and with less integrations and command-line tools, its has way beefier machines, easily several times faster than what travis offers even to paid customers. And their tools are improving. Keep up the good work.
- wahnfrieden 9y agoWe did not pursue Codeship due to lack of a "codeship.yml"-style configuration so that we could track config in source control.
- moritzplassnig 9y agoCodeship founder here. We support .yml file config for our container/Docker based product (called Codeship Pro). We're looking into offering a UI + yml file approach for both products because we see a 50-50 split when it comes to what people prefer. Thanks for the feedback :)
- jrudolph 9y agoHeroku is awesome for this use case, but there are also great alternatives based on Cloud Foundry. Cloud Foundry is an open source PaaS with support for Heroku buildpacks + docker containers. At a median of ~5€/GB RAM per month they will also be a bit cheaper than Heroku. Some providers that offer Cloud Foundry including a free tier for small Apps: - Meshcloud (hosted in Germany, 2 Locations) https://panel.meshcloud.io/#/register https://panel.meshcloud.io/#/register (Disclaimer: I'm a founder there) - Pivotal Web Services (hosted on AWS US): https://run.pivotal.io/ https://run.pivotal.io/ - IBM Bluemix https://www.ibm.com/cloud-computing/bluemix/ https://www.ibm.com/cloud-computing/bluemix/ (various locations, a bit more expensive than the other two though)
- NightMKoder 9y agoOnce you need something a bit more heavyweight than Travis (e.g. more CPU) I would highly suggest Buildkite (https://www.buildkite.com https://www.buildkite.com). It’s a “bring your own machines” solution, which means you can have beefy machines (e.g. via buildkite on EC2 spot autoscaling), or even machines that run inside production without handing buildkite itself any secrets. We’ve been using it at Ladder and it’s been phenomenal. Also it has a lot of emojis.
- vacri 9y agoAnother very happy buildkite user here. It's extremely flexible and the crew just keep adding feature after feature.
- nicodjimenez 9y ago+1 for mentioning Buildkite, also works great for machine learning
- mopierotti 9y agoIs there a particular feature that's suited to machine learning? I'm having trouble understanding the connection.
- spinlock 9y agothe way one process can feed into the next. you can setup data pipelines that are nice for ML projects.
- onekiloparsec 9y agoAn alternative to travis-ci which also works well: circleci.com. It has a free tier (whatever the open/close status).
- renw0rp 9y agoany suggestions for self hosted stack?
- brodock 9y agoInstead of Slack -> Mattermost or Rocket.Chat Sentry also has self-hosted option available (and it's also opensource). Instead of GitHub + Travis CI, you can use GitLab (it also has bundled CI capability, based on Docker). with GitLab, you can also have Continuos Deployment, wired up with the CI solution. The best experience will be deploying to Kubernetes, as it will give you even monitoring, automatically, but you can code custom deployment logic as a step in the CI pipeline.
- peterwwillis 9y ago[ 'GitLab', 'Jenkins', 'Vagrant || OpenShift[.org]', 'Errbit', 'IRCd || ejabberd' ] Just keep in mind that "self-hosted" means "self-supported". I hope you pay yourself well.
- planetjones 9y agoNot sure bitbucket pipelines are mentioned here. But for free I manage to commit Hugo source files to a private git repository, generate the static files by spinning up a docker container and generating them using Hugo and finally rsyncing the static output to my Webserver. All for free. You only get 50 minutes of time now (for free) in the docker instance but for simple continuous deployment cases like mine it’s very effective.
- james-skemp 9y agoYou might want to look at Gitlab if you want to drop the need for a web server. Their pages support supports Hugo much better than GitHub (which leans toward Jekyll), and is pretty well documented.
- acejam 9y agoThe last time I tried using pipelines, it wouldn't let me use any arbitrary Docker image, I had to use one of the Atlassian "blessed" images. :/
- desireco42 9y agoAdd Netlify and it will continue to be free. I use Hugo as well, Netlify compiles Hugo on it's own, so I don't need workflows, but it is cool to have them.
- Domenic_S 9y agoWhy is that better than generating static locally? Seems like a lot of fiddling for no benefit.
- planetjones 9y agoIt’s fully automated. I do a commit and push from any machine that I have cloned the repo to and it gets published automatically. Now I don’t need to download Hugo, remember the Hugo commands, remember my web server login details, etc. I think it’s a lot less fiddling as all I need to remember now is how to add a file to a git repo to get a post published.
- jonatanheyman 9y agoA great self-hosted CI alternative is Drone.io (https://github.com/drone/drone https://github.com/drone/drone). To set it up is simply a matter of starting a Docker image and creating a .drone.yml file (very similar to .travis.yml). Note: I'm not affiliated with the project in any way. I'm just a happy user who think the project deserves way more buzz.
- bfirsh 9y agoI have a similar setup, but using Docker images instead of Heroku buildpacks. Buildpacks are horrible if you want to do anything custom. To make this work, I use Codeship instead of Travis. Codeship builds a Docker image, runs the tests against the image, then pushes the image to Heroku if the tests pass: https://documentation.codeship.com/pro/continuous-deployment/heroku/ https://documentation.codeship.com/pro/continuous-deployment...
- grogenaut 9y agoDoesn't show any automated verification of code in deployed fashion, no load tests or verifications which would be part of a world class pipeline. Also no canary or red-blue tree type rollout hooked to sentry. At least it has a staging but don't know what they're using it for other than to test migrations before promoting to prod. Nice little write up but far from world class.
- sublimino 9y agoExactly - continuous deployment is about moving some/most testing to production, not eliminating it altogether. Acceptance testing the system once deployed is the cornerstone of organisational acceptance of the CD doctrine - errors will always happen, it's just that they can now be fixed much faster in production. Using an acceptance test suite as a deployment safety harness should dramatically increase the quality of tests, and hopefully the application too.
- jfaucett 9y agoI can't believe the author doesn't mention gitlab. Setting up full CI on gitlab is amazing. They have the easily integratable gitlab runners i.e. travis-ci functionality + different build stages for automatic deployment via ansible/chef/whatever and unlimited free private projects to boot. If you're working on anything that can't be public I highly recommend gitlab. Its an amazing service. FYI: I am in no way affiliated with Gitlab.
- simonw 9y agoI haven't tried it :) I'll definitely give it a go.
- james-skemp 9y agoI think I saw that GitHub now supports non-gh_pages branches, but Gitlab pages are also much easier to setup. I've moved all my sites to Gitlab pages generated via Hugo or Jekyll. Uptime (via Uptime Robot) just doesn't seem as good as GitHub, but it's not bad.
- y4mi 9y ago> , but Gitlab pages are also much easier to setup uuh, what? i think gitlab pages are wonderful, because you're not restricted to jekyll, but how is it easier to setup? you only need to toggle the checkpoint in the repository settings...
- james-skemp 9y agoSorry, easier to setup because you're not restricted to Jekyll/can easily use Hugo/etcetera. Edit: And what I said in original comment. Gh_pages branch was required, but Gitlab could just use the built assets.
- acejam 9y agoGitLab is great and has come a very long way over the last several years. However, I still gladly pay GitHub $7 a month for my private repos because I feel that GitHub truly does offer a better code review and repository experience. GitLab in certain places is still very rough around the edges, not to mention performance has always been a concern. To me, GitHub still is the gold standard and will likely be for quite some time.
- desireco42 9y agoI like Bitbucket better for my personal private repo's. Also would like to mention few more tools I like, Netlify for static site hosting, SemaphoreCI as alternative to Travis, I think both are good.
- machbio 9y agoThis Talks about Freehosted tools on Code that is Opensource - for Closed source - this does not work.
- markogrady 9y agoI have started using visual studio team services, it is really good and it's free. https://www.visualstudio.com/team-services/ https://www.visualstudio.com/team-services/ I used to use team city, octopus and jenkins in previous jobs. Personally, it is really easy to use and you don't have to use the microsoft stack. It supports Python, Ruby, node etc... It feels to me like they waited to see all the good bits people have done and then just implemented them. I didn't want to manage a server so it is perfect for me.
- kkapelon 9y ago>The integration of Travis with GitHub runs deep At the moment there are 10+ companies that do the exact same thing. http://blog.codepipes.com/hosted-ci-comparison/hosted-ci-comparison.html http://blog.codepipes.com/hosted-ci-comparison/hosted-ci-com... If you don't have time to click the link: - Buddy Works - CircleCI - Codefresh - Codeship - DeployBot - Distelli - SemaphoreCI - Shippable - SolanoLabs - Vexor - Wercker
- manigandham 9y agohttps://buddy.works/ https://buddy.works/ is great, fast builds, slick UI, and a free self-hosted enterprise version.
- AdeptusAquinas 9y agoYou can also do every step listed, for free using a single platform: VSTS (Visual Studio Team Services). Online, free for small teams, Git source control, CI/build/release, supports every major build stack, not just MS. It even has a top quality agile work tracker, that I've found better (or at least more balanced) than Jira/Trello equivs. EDIT: link: https://www.visualstudio.com/team-services/ https://www.visualstudio.com/team-services/
- luord 9y agoI use gitlab instead of GitHub and Travis and tend to use the free tiers of GAE and Heroku indistinctly. But man, I wasn't even aware of something like Sentry existing, I'll have to get me some of that. I also need to think of the messaging system. I'll probably start using mattermost instead of slack, though; because GitLab.
- tarantor 9y agoI would prefer Jenkins instead of TravisCI. And bitbucket provides private repos and I always prefer BitBucket or even Gitlab but no GitHub.
- daneah 9y agoI still have my stuff on AWS instead of Heroku and use Rollbar instead of Sentry (not for any specific reason), but overall I'm really happy to hear that I'm tracking well with this setup. I find it smooth as butter most days. Thanks so much for sharing!