3 ms·

Why Continuous Deployment?

57 points by peter123 17y ago

russell 17y ago
Continuous Deployment is a very intriguing idea. I haven't tried it, but in the the limit it makes sense. A few years ago I worked for a company that did large batch releases, scheduled every 3 months. The first month was spent negotiating with product management over product requirements and writing functional specifications. The next month was writing code. The third month was fairly rigorous QA, so much so that there was usually another month to fix the problems that QA found. The breakdown was 25% design documentation, 25% programming, and 50% fixing bugs that should have been caught in programming. The quality of the code that made it to production was very good, except for one time where a single bug brought the company to its knees for a week. More recently, I did a consulting gig a company that released every two weeks with a one week QA cycle that allowed for a round of critical bug fixes. This was much smoother. There were rarely any show stoppers and even those were so fresh in eveyone's minds that they were quickly fixed. Neither place practiced TDD or even rigorous unit testing. Testing was pretty much walking through manual scripts. It seems to me that continuous integration to production requires TDD or something close to it. I also question whether it is appropriate for an application with fiduciary responsibility, but I would love to be convinced otherwise.
- russell 17y ago
  EDIT: I found a howto page that answers some of my questions and shows hw it works, http://radar.oreilly.com/2009/03/continuous-deployment-5-eas.html http://radar.oreilly.com/2009/03/continuous-deployment-5-eas...
- abstractbill 17y ago
  I don't practice TDD or unit testing, but I deploy code on average more than 5 times every day. Despite not doing TDD or unit testing, I have enough confidence in my code that I spent a month in Greece on honeymoon last year with only a Blackberry, while an average of about 50,000 concurrent users were connected to the chat system I wrote for justin.tv. In that month the system didn't have a single problem. My "secret", if I have one, is that every change I release is very small (maybe 5 or 10 lines of code - 20 lines at the very most). I read and re-read my changes thoroughly before I deploy, and test the new behavior many times. The evolution of the chat system I wrote has been a huge number of very small steps.
  - DanielBMarkham 17y ago
    I'm curious -- were you using a functional language? An OOP language? Something else?
    - abstractbill 17y ago
      Python, because of the twisted library - if I was rewriting, I'd probably use Clojure, but at the time I wrote the system there was no widely-used Lisp with an also widely-used non-blocking i/o library.
  - biohacker42 17y ago
    and test the new behavior many times Manually? You can save a lot of time if automate that.
    - abstractbill 17y ago
      I do have non-unit tests (but only for things that have actually failed more than once). Specifically, I have a bunch of chat bots that periodically log in to the production chat servers, talk to each other, ban each other, etc, etc, and make sure they get the responses they expect. Yes, new functionality I test manually. I believe the time taken to write tests for things that have never failed is wasted.
      - axod 17y ago
        Agreed. Given the choice of logging or unit testing, logging is probably 100x as important in finding and fixing bugs.
      - Afton 17y ago
        I believe the time taken to write tests for things that have never failed is wasted. Yes yes yes. I've seen a lot of tests that, if they failed, would have meant that a thousand other tests would have also failed, and that test would have been simply more noise.
mrduncan 17y ago
A post from about 5 months ago going into some detail on IMVUs continuous deployment process which may be interesting to some: http://news.ycombinator.com/item?id=475017 http://news.ycombinator.com/item?id=475017
richcollins 17y ago
The interesting idea that Eric introduces here is the concept of deploying plumbing for big new features continuously. This lets you work and small batches while making big changes.
- blader 17y ago
  Insightful observation.
zacharypinter 17y ago
I like continuous deployment and want to use it on a few websites I run. However, it seems to depend on a large user set in order to "test deploy" the application to a subset of the user base. Any thoughts on using this for projects with a small (< 1000) user base?
- eries 17y ago
  I think the principles apply, but the specific practice may have to change a bit. If you look at the five-part "cluster immune system" I normally use, only one part is dependent on a large user base, and that is incremental deploy. You can still use all the automated testing, sandbox, continuous integration and real-time alerting practices just as well. I would suggest using something like Five Whys to figure out how different your situation really is, rather than assume you'll have a lot of problems due to a small user base: http://startuplessonslearned.blogspot.com/2008/11/five-whys.html http://startuplessonslearned.blogspot.com/2008/11/five-whys.... You might find that the smaller customer base is actually a big advantage, since it allows you personally get to know many more customers and therefore find it easier to find and debug problems in realtime. Just a thought.
  - trapper 17y ago
    For this we thought about recording user input, then replaying it on the new build as a pre-deploy step and looking for errors in logs etc. Obviously this would only work without major ui/schema changes. Have you tried anything like this?

57 points by peter123 17y ago

Continuous Deployment is a very intriguing idea. I haven't tried it, but in the the limit it makes sense. A few years ago I worked for a company that did large batch releases, scheduled every 3 months. The first month was spent negotiating with product management over product requirements and writing functional specifications. The next month was writing code. The third month was fairly rigorous QA, so much so that there was usually another month to fix the problems that QA found. The breakdown was 25% design documentation, 25% programming, and 50% fixing bugs that should have been caught in programming. The quality of the code that made it to production was very good, except for one time where a single bug brought the company to its knees for a week. More recently, I did a consulting gig a company that released every two weeks with a one week QA cycle that allowed for a round of critical bug fixes. This was much smoother. There were rarely any show stoppers and even those were so fresh in eveyone's minds that they were quickly fixed. Neither place practiced TDD or even rigorous unit testing. Testing was pretty much walking through manual scripts. It seems to me that continuous integration to production requires TDD or something close to it. I also question whether it is appropriate for an application with fiduciary responsibility, but I would love to be convinced otherwise.

russell 17y ago
EDIT: I found a howto page that answers some of my questions and shows hw it works, http://radar.oreilly.com/2009/03/continuous-deployment-5-eas.html http://radar.oreilly.com/2009/03/continuous-deployment-5-eas...
abstractbill 17y ago
I don't practice TDD or unit testing, but I deploy code on average more than 5 times every day. Despite not doing TDD or unit testing, I have enough confidence in my code that I spent a month in Greece on honeymoon last year with only a Blackberry, while an average of about 50,000 concurrent users were connected to the chat system I wrote for justin.tv. In that month the system didn't have a single problem. My "secret", if I have one, is that every change I release is very small (maybe 5 or 10 lines of code - 20 lines at the very most). I read and re-read my changes thoroughly before I deploy, and test the new behavior many times. The evolution of the chat system I wrote has been a huge number of very small steps.
- DanielBMarkham 17y ago
  I'm curious -- were you using a functional language? An OOP language? Something else?
  - abstractbill 17y ago
    Python, because of the twisted library - if I was rewriting, I'd probably use Clojure, but at the time I wrote the system there was no widely-used Lisp with an also widely-used non-blocking i/o library.
- biohacker42 17y ago
  and test the new behavior many times Manually? You can save a lot of time if automate that.
  - abstractbill 17y ago
    I do have non-unit tests (but only for things that have actually failed more than once). Specifically, I have a bunch of chat bots that periodically log in to the production chat servers, talk to each other, ban each other, etc, etc, and make sure they get the responses they expect. Yes, new functionality I test manually. I believe the time taken to write tests for things that have never failed is wasted.
    - axod 17y ago
      Agreed. Given the choice of logging or unit testing, logging is probably 100x as important in finding and fixing bugs.
    - Afton 17y ago
      I believe the time taken to write tests for things that have never failed is wasted. Yes yes yes. I've seen a lot of tests that, if they failed, would have meant that a thousand other tests would have also failed, and that test would have been simply more noise.

mrduncan 17y ago

A post from about 5 months ago going into some detail on IMVUs continuous deployment process which may be interesting to some: http://news.ycombinator.com/item?id=475017 http://news.ycombinator.com/item?id=475017

richcollins 17y ago

The interesting idea that Eric introduces here is the concept of deploying plumbing for big new features continuously. This lets you work and small batches while making big changes.

blader 17y ago
Insightful observation.

zacharypinter 17y ago

I like continuous deployment and want to use it on a few websites I run. However, it seems to depend on a large user set in order to "test deploy" the application to a subset of the user base. Any thoughts on using this for projects with a small (< 1000) user base?

eries 17y ago
I think the principles apply, but the specific practice may have to change a bit. If you look at the five-part "cluster immune system" I normally use, only one part is dependent on a large user base, and that is incremental deploy. You can still use all the automated testing, sandbox, continuous integration and real-time alerting practices just as well. I would suggest using something like Five Whys to figure out how different your situation really is, rather than assume you'll have a lot of problems due to a small user base: http://startuplessonslearned.blogspot.com/2008/11/five-whys.html http://startuplessonslearned.blogspot.com/2008/11/five-whys.... You might find that the smaller customer base is actually a big advantage, since it allows you personally get to know many more customers and therefore find it easier to find and debug problems in realtime. Just a thought.
- trapper 17y ago
  For this we thought about recording user input, then replaying it on the new build as a pre-deploy step and looking for errors in logs etc. Obviously this would only work without major ui/schema changes. Have you tried anything like this?