7 ms·

Thanks for creating software which has been an immense service to the community, and which I rely on quite a bit. Tangent mode on: Somebody really, really nee

by patio11 13y ago

Thanks for creating software which has been an immense service to the community, and which I rely on quite a bit. Tangent mode on: Somebody really, really needs to write the How To Deploy Rails Without Losing Your Sanity handbook. I will buy a copy. It will sell thousands of them. A lot of the problems with people's interactions with Capistrano are environment/ops problems which have known solutions that work, but which rely on people having a great understanding of arcane trivia which is spread across conference presentations, blog posts, commit messages, and the practical experience of the best Rails teams. Unless you're prepared for an archaeological expedition every time you start a new Rails project, you're going to do something wrong. You should see the bubblegum and duct tape which I came up with, and it mostly works, but I know it is bubblegum and duct tape. Example: Non-deterministic deploys of code from (usually) un-tagged source control I feel lucky in that I was mentored by an engineer who decided to teach me, one day, Why We Tag Shit. But for the Why We Tag Shit discussion, I would be like almost every other intermediate Rails engineer, and be totally ignorant of why that was a best practice until lack of it bit me in the keister, at which point the server is down and one has to rearchitect major parts of the deployment workflow to do things the right way. Why We Tag Shit is only about a 500 word discussion, but it's one piece of organic knowledge of the hundreds you need to do things right, and it is (to the best of my knowledge) not covered in docs/QuickStarts/etc because that seems to be out of the purview of the framework proper (I guess?). I'm sure that I'm ignorant of several of the hundreds of pieces of things one needs to do to do deployment right, as evidenced by my fear every time I execute my deploy scripts. I, and I must assume many other companies, am willing to pay for an option which gets me to a non-bubblegum and duct tape outcome. Seriously, folks: there is a product here.

raverbashing 13y ago
I'll be downvoted by the "newer generation" but here's a pet peeve of mine The same reason "why we tag shit" is the reason downloading packages from gem/pip is not enough Who guarantees that when you do your deploy, the library you need is there? How many times did your build break because it had a glitch downloading the package? Keep a way of rebuilding and deploying your software. All of it
- DrJ 13y ago
  for High Availability you would be a fool not to either run your own mirror, or package everything you need into a single bundle.
- sjtgraham 13y ago
  > How many times did your build break because it had a glitch downloading the package? This happens very infrequently in my experience, but it's simple on most CI servers to restart a build.
  - lamby 13y ago
    I'm pretty sure that wasn't the point - however infrequently it occurs, it violates the principle that only changes introduced since the previous build could have made it fail. In practical terms, this means that you never know whether change X is faulty or simply the world is temporarily faulty. (Of course, this may or may not be not useful to you to know.)
    - sjtgraham 13y ago
      True, but a build could fail arbitrarily fail for any unexpected reason. Pragmatism is important, I'd avoid adding an inconvenient step to my build/deploy process just to mitigate an infrequent edge case.
      - byroot 13y ago
        Infrequent (I don't remeber that it happened to me during the hundred or thousand deploy I made with capistrano) and mostly harmless. If the deploy fail for any reason, it's aborted, you just have to run it again.
        pyre 13y ago
        What do you do if the package servers go down for a couple of days? "Oops! Guess I can't deploy. Oh well!"
sjtgraham 13y ago
Capistrano, what a Byzantine way to deploy software; I haven't typed cap deploy in about 3 years. Heroku's model of Git-based deployment and 12-factor apps has never given me a problem. Also Heroku deploys are de facto tagged as Releases (https://devcenter.heroku.com/articles/releases https://devcenter.heroku.com/articles/releases). I can't wait for Flynn (http://flynn.io/ http://flynn.io/) to be released, it provides a Heroku-esque Git-based deployment strategy on any host. P.S. I enjoy reading your comments as do many others, but I feel you hand-wave over important points sometimes, e.g. Why you feel tagging is important. If it truly is a 500 word discussion it shouldn't be hard to synopsise.
- patio11 13y ago
  It is tangential to the point of the comment, since this only gets you to 0.001% of a Rails deployment which works, but if you're interested: Tags, plus sufficiently advanced dependency management and database migrations (n.b. not trivial!), allow you to track and reproduce past states of the deployed software. For example: A customer reports a problem. You verify the problem exists. The customer reports that the system didn't have the problem last week Tuesday. If you want to know why, tagged releases (and a record of them -- incidentally, another one of the 643 things to know is "All deploys should be recorded somewhere") let you travel in time back to the software in use last Tuesday, verify if the customer's recollection is accurate (well, sorta), and then compare just what has changed rather than trying to run down the bug without any historical context. Naming things also lets you talk about them, which is helpful, because you'll want to talk about individual releases of the software. (I mean, sure, you could use random hexadecimal numbers, but for whatever reason people find production_release_20131007_3 a bit more informative than 066630de00d242137efab6bf21b8ea04aeee7a1d. One glance at the first one tells you that it's the 3rd deploy from October 7th, assuming you've used the company's naming convention for more than a minute. The second one tells you nothing and is difficult to pull useful information out of without having to manually cross-tabulate it with you git repository, which would be unfortunate if it were e.g. attached to a log file, customer support request, or Wiki article about known-good releases for interacting with external systems.)
  - sjtgraham 13y ago
talkingquickly 13y ago
I'm about a third of the way through writing exactly that, covering deployment (capistrano) and basic server provisioning (chef), specifically for Rails apps. If you (or anyone) is interested my email/ twitters in my profile. Aiming to have it out end of 2013/ start of 2014 but would be happy to release it to a few people early for feedback.
jessepollak 13y ago
you don't happen to have a good online reference on Why We Tag Shit, do you?
timdorr 13y ago
There definitely is a product here, and it's already being built. Docker is the underlying framework for which we'll see significant improvement for deploys in the near future. In the meantime, and if you're on AWS, check out OpsWorks. It's the cat's pajamas. It both makes much of the hassle of deploying a thing of the past, is extremely flexible, and makes it possible to have Heroku-like "git push" deploys without giving up control (or being forced into Beanstalk's opinionated ways). Amazon made a very smart purchase when they bought Scalarium.
davedx 13y ago
This is one of the biggest gotchas of (some of) the high productivity full stack frameworks that are out there, in my experience. I've also had issues getting Meteor deployed, and while it's in many ways a smaller, simpler beast than Rails, it has the same kind of black magic and hitting-a-wall-at-100mph deployment experience. Both Rails and Meteor will let you develop apps at breakneck speed, with their powerful, magic cores and their excellent module ecosystems, but when you get to deploying them you'll sometimes wish you'd stayed on LAMP or (god forbid) ASP.NET. Yes, I really said that.
- gesman 13y ago
  I glad you said that. I'm seriously considering moving from RoR to ASP.NET/C# platform. We're spending $50k-$100k/yr on Linux system administration of all the small and big moving parts that always fall apart and requires constant babysitting. I want to give MSFT all this money and use MSVS, C#, ASP.NET and Azure for everything. I love MSVS and always loves MSVC (back in the days). I love performance of C{++|#}. I don't feel inspired to build great SaaS solutions with shitty interpreted languages, forcing to use made-in-the-basement plugins (or gems) and all that trailing crap of sluggishness, upgrades, fixes and weird behaviors that follows. Now you all may downvote me!
robmclarty 13y ago
Not exactly a product, nor wholly Capistrano-centric, but I recently put together an article on how I deploy Rails on a new VM from scratch: http://robmclarty.com/blog/how-to-deploy-a-rails-4-app-with-git-and-capistrano http://robmclarty.com/blog/how-to-deploy-a-rails-4-app-with-...

by patio11 13y ago

raverbashing 13y ago

I'll be downvoted by the "newer generation" but here's a pet peeve of mine The same reason "why we tag shit" is the reason downloading packages from gem/pip is not enough Who guarantees that when you do your deploy, the library you need is there? How many times did your build break because it had a glitch downloading the package? Keep a way of rebuilding and deploying your software. All of it

DrJ 13y ago
for High Availability you would be a fool not to either run your own mirror, or package everything you need into a single bundle.
sjtgraham 13y ago
> How many times did your build break because it had a glitch downloading the package? This happens very infrequently in my experience, but it's simple on most CI servers to restart a build.
- lamby 13y ago
  I'm pretty sure that wasn't the point - however infrequently it occurs, it violates the principle that only changes introduced since the previous build could have made it fail. In practical terms, this means that you never know whether change X is faulty or simply the world is temporarily faulty. (Of course, this may or may not be not useful to you to know.)
  - sjtgraham 13y ago
    True, but a build could fail arbitrarily fail for any unexpected reason. Pragmatism is important, I'd avoid adding an inconvenient step to my build/deploy process just to mitigate an infrequent edge case.
    - byroot 13y ago
      Infrequent (I don't remeber that it happened to me during the hundred or thousand deploy I made with capistrano) and mostly harmless. If the deploy fail for any reason, it's aborted, you just have to run it again.
      - pyre 13y ago
        What do you do if the package servers go down for a couple of days? "Oops! Guess I can't deploy. Oh well!"

sjtgraham 13y ago

Capistrano, what a Byzantine way to deploy software; I haven't typed cap deploy in about 3 years. Heroku's model of Git-based deployment and 12-factor apps has never given me a problem. Also Heroku deploys are de facto tagged as Releases (https://devcenter.heroku.com/articles/releases https://devcenter.heroku.com/articles/releases). I can't wait for Flynn (http://flynn.io/ http://flynn.io/) to be released, it provides a Heroku-esque Git-based deployment strategy on any host. P.S. I enjoy reading your comments as do many others, but I feel you hand-wave over important points sometimes, e.g. Why you feel tagging is important. If it truly is a 500 word discussion it shouldn't be hard to synopsise.

patio11 13y ago
It is tangential to the point of the comment, since this only gets you to 0.001% of a Rails deployment which works, but if you're interested: Tags, plus sufficiently advanced dependency management and database migrations (n.b. not trivial!), allow you to track and reproduce past states of the deployed software. For example: A customer reports a problem. You verify the problem exists. The customer reports that the system didn't have the problem last week Tuesday. If you want to know why, tagged releases (and a record of them -- incidentally, another one of the 643 things to know is "All deploys should be recorded somewhere") let you travel in time back to the software in use last Tuesday, verify if the customer's recollection is accurate (well, sorta), and then compare just what has changed rather than trying to run down the bug without any historical context. Naming things also lets you talk about them, which is helpful, because you'll want to talk about individual releases of the software. (I mean, sure, you could use random hexadecimal numbers, but for whatever reason people find production_release_20131007_3 a bit more informative than 066630de00d242137efab6bf21b8ea04aeee7a1d. One glance at the first one tells you that it's the 3rd deploy from October 7th, assuming you've used the company's naming convention for more than a minute. The second one tells you nothing and is difficult to pull useful information out of without having to manually cross-tabulate it with you git repository, which would be unfortunate if it were e.g. attached to a log file, customer support request, or Wiki article about known-good releases for interacting with external systems.)
- sjtgraham 13y ago

talkingquickly 13y ago

I'm about a third of the way through writing exactly that, covering deployment (capistrano) and basic server provisioning (chef), specifically for Rails apps. If you (or anyone) is interested my email/ twitters in my profile. Aiming to have it out end of 2013/ start of 2014 but would be happy to release it to a few people early for feedback.

jessepollak 13y ago

you don't happen to have a good online reference on Why We Tag Shit, do you?

timdorr 13y ago

There definitely is a product here, and it's already being built. Docker is the underlying framework for which we'll see significant improvement for deploys in the near future. In the meantime, and if you're on AWS, check out OpsWorks. It's the cat's pajamas. It both makes much of the hassle of deploying a thing of the past, is extremely flexible, and makes it possible to have Heroku-like "git push" deploys without giving up control (or being forced into Beanstalk's opinionated ways). Amazon made a very smart purchase when they bought Scalarium.

davedx 13y ago

This is one of the biggest gotchas of (some of) the high productivity full stack frameworks that are out there, in my experience. I've also had issues getting Meteor deployed, and while it's in many ways a smaller, simpler beast than Rails, it has the same kind of black magic and hitting-a-wall-at-100mph deployment experience. Both Rails and Meteor will let you develop apps at breakneck speed, with their powerful, magic cores and their excellent module ecosystems, but when you get to deploying them you'll sometimes wish you'd stayed on LAMP or (god forbid) ASP.NET. Yes, I really said that.

gesman 13y ago
I glad you said that. I'm seriously considering moving from RoR to ASP.NET/C# platform. We're spending $50k-$100k/yr on Linux system administration of all the small and big moving parts that always fall apart and requires constant babysitting. I want to give MSFT all this money and use MSVS, C#, ASP.NET and Azure for everything. I love MSVS and always loves MSVC (back in the days). I love performance of C{++|#}. I don't feel inspired to build great SaaS solutions with shitty interpreted languages, forcing to use made-in-the-basement plugins (or gems) and all that trailing crap of sluggishness, upgrades, fixes and weird behaviors that follows. Now you all may downvote me!

robmclarty 13y ago

Not exactly a product, nor wholly Capistrano-centric, but I recently put together an article on how I deploy Rails on a new VM from scratch: http://robmclarty.com/blog/how-to-deploy-a-rails-4-app-with-git-and-capistrano http://robmclarty.com/blog/how-to-deploy-a-rails-4-app-with-...