12 ms·

Maybe we’ve not improved things as much as we think we have with Devops?

47 points by michaelbiven 11y ago

PaulHoule 11y ago
One problem is that existing devops tools are not that good. For instance it seems insane to me that packer and vagrant exist as entirely separate products. As a java programmer I find that vagrant files are way too verbose. I am sure I could write some Ruby functions to take care of that and also to fix zillions of other things that are awful about Vagrant, but then everybody writes their own functions.
- toomuchtodo 11y ago
  Try Docker instead. I can build Docker containers that provide our devs with a full blown dev environment that mimics production, which anyone on the team can then pull down with a simple "docker pull <registry>/<container name>:latest" command.
  - falcolas 11y ago
    Docker has its own issues, and in the end is remarkably similar to Vagrant when it comes to development. Docker's biggest win on the development (not deploy) front is that docker containers are faster to set up and tear down; though with a decent Vagrant setup and a package cache, you can build a brand new Vagrant image in under a minute. Since I'm on a Mac, and thus going to be running VirtualBox anyways, I prefer to just work with the VMs directly.
- xacaxulu 11y ago
  Vagrant is a very well developed tool and has tons of configurations available in the Vagrantfile. It does take a lot of time to get deep into it however. When coupled with Chef's Test Kitchen, vagrant is a powerful tool. Side note; they actively review and accept pull requests.
- rev_bird 11y ago
  The Vagrant files we've been using have been whittled down to almost nothing -- we use Chef to provision boxes, so the vagrantfile is pretty much just "install Chef, forward ports, run role xxx." At that point, it just pulls the role (and cookbooks) out of the user's local chef repo -- and getting the Vagrant Berkshelf plugin running even removes the need to deal with downloading cookbooks at all.
- jtreminio 11y ago
  You're not supposed to use Vagrant for the actual provisioning of a box - use the proper tools like Puppet, Chef, Ansible, Salt, etc. And Packer is for generating the base boxes. Packer abstracts away the virtual machine software like Virtualbox, VMWare, Parallels, etc.
davexunit 11y ago
Unfortunately, developing and deploying web applications is still a total nightmare. To build anything useful you'll need about four distinct package managers to get all of the necessary dependencies and some mish-mash of configuration management and provisioning tools to develop it, and perhaps a different set of configuration management and provisioning tools to deploy it. We can do better than this!
- deleted 11y ago
  [deleted]
- danudey 11y ago
  Hmm, let's see… > To build anything useful you'll need about four distinct package managers to get all of the necessary dependencies * Ubuntu packages (dpkg via apt) * Python packages (via pip) * Our own software packages (via our own dependency manager) * Custom-built software packages (e.g. nginx) or non-package software installations (e.g. activemq) > some mish-mash of configuration management and provisioning tools to develop it We use salt for (some of) our configuration management along with just symlinks into git repositories (where possible, e.g. MySQL and others won't follow symlinks for security) > perhaps a different set of configuration management and provisioning tools to deploy it We have a custom-built deployment system which we use to deploy git packages (including the configuration packages that the configuration files are symlinked into) as well as our actual software. So… yeah. You pretty much hit the nail on the head.
  - davexunit 11y ago
    Thanks for the real life example.
rjbwork 11y ago
Y'know what? Up until Jan last year I had never even touched a server to deploy an app, much less set up a CI/CD/DevOps environment. Now, I'm comfortable with TeamCity, Octopus Deploy, Azure (a ton of hosted services), PowerShell, and a host of other tools. I've set up bunch of HipChat integrations so we know exactly where in our pipleline a particular build is (code pushed, dev build/deploy, and promotions to various environments), if there was a problem with a unit test run, if there were problems with integration tests, and if there are problems in dev/staging/production with our actual live running code via HipChat integrations with our structured log processing server. There are good tools out there and they're pretty usable - just maybe not at your price-point of free for your environment. I went from 0 to 1-click whole-environment deployments in just over a year.
- moserware 11y ago
  It's nice to hear of other people with a similar stack on HN (Azure, PowerShell, TeamCity, HipChat, and presumably C#/.NET)! It's working well for us too. In addition, we're increasingly doing more with ChatOps with mmbot (as an Azure Web Job): https://github.com/mmbot/mmbot https://github.com/mmbot/mmbot One nice thing about mmbot is that if you use the DLL mode instead of the CSX mode, you get a great IDE debugging experience which makes it even faster to create meaningful ChatOps extensions. I've found it to be a more pleasant development experience that Hubot with CoffeeScript.
  - rjbwork 11y ago
    Hah, we also use mmBot, but while it was our original plan to have it be our "DevOps 1 stop shop" we ended up shying away from that idea in favor of a more hands off approach. We still use our mmBot for a lot of back-office querying type operations, but decided that, at least for the moment, TC/Octopus will be our way forward.
- mordocai 11y ago
  I don't know if you read the same article I did. The author specifically said that we'd come a long way on tools but not made much headway on the people issues related to Dev/Ops. Your comment does not appear to be adding anything to the article besides another anecdote that the tools are good (and if that is what you meant it to be, then I misread the tone of your comment).
Spooky23 11y ago
Of course we haven't. In many cases, DevOps == have developers push code to production platforms. IMO, there's been a lack of attention to the engineering side of the platforms themselves. In the places that I've seen that yammer on about devops, I see a lot of agile projects where version 1.0 works great, but the ongoing maintenance of the platform (whatever it is) is a complete nightmare.
sgoings 11y ago
I think it's worth reflecting that not every improvement initiative (in this case: see DevOps) across the whole world is moving as one large entity. The ending line: "Otherwise in my opinion it feels more like we’re treading water and not making the advancements we could be making." sounds like an existential fallacy to me - it has been my experience that although the "DevOps vision" has not changed significantly, the actual practices and evangelism of DevOps have been infiltrating and transforming the way organizations think about software engineering.
- sanderjd 11y ago
  Basically my thoughts: 5 years is not a very long time! It isn't treading water, it's taking time to mature and penetrate. It's actually taking root surprisingly quickly, and only looks slow from an early-adopter's perspective. If those talks from 2010 and 2011 were given to substantially the exact same audience as that talk from 2014, then yeah, maybe I see the point that things are just being re-hashed with little progress, but I doubt that's the case. Giving the same exact talks to an ever-expanding audience over 5 years is already a win.
deleted 11y ago
[deleted]
whistlerbrk 11y ago
This recent trend of articles wrt to devops it making me want to post here. Not trying to be too snarky, but why can't people just use shell scripts for automation? They are easy to repair, in a language you (should) know, reproducible, and testable. They are native to your target, supremely portable, etc, etc.
- cheeseprocedure 11y ago
  Because it's not worth re-implementing the functionality available in Jenkins, Chef, etc. etc. etc.
  - stonogo 11y ago
    This is a misunderstanding of the situation. Jenkins specifically has no business being anywhere near a production deployment; it is a development tool, despite "devops" people tending to use it as a web-interfaced crond. Automation should be tooled to the production environment by people who are familiar with the production requirements and -- most importantly -- by the people who will be expected to maintain the environment and respond to critical issues.
    - boon 11y ago
      What's wrong with using jenkins to invoke the local tooling?
    - rev_bird 11y ago
      > Jenkins specifically has no business being anywhere near a production deployment I'm curious about this -- what's wrong with using the same deployment tool for running dev _and_ the other environments? Jenkins itself is a life-saver for, like you said, doing development builds/testing/reporting, but if we've got one script that deploys an image to production, why not trigger it with a Jenkins job, if only to keep everything centralized and maintain coherent deployment histories?
  - nickysielicki 11y ago
    Why do you think most people don't use cfengine?
xj9 11y ago
I'm really looking forward to Triton[1] coming out of beta. This may be unpopular/wrong, but I don't think you should have to care about what machine your app is running on in any way. [1]: https://www.joyent.com/developers/triton-faq https://www.joyent.com/developers/triton-faq

47 points by michaelbiven 11y ago

One problem is that existing devops tools are not that good. For instance it seems insane to me that packer and vagrant exist as entirely separate products. As a java programmer I find that vagrant files are way too verbose. I am sure I could write some Ruby functions to take care of that and also to fix zillions of other things that are awful about Vagrant, but then everybody writes their own functions.

toomuchtodo 11y ago
Try Docker instead. I can build Docker containers that provide our devs with a full blown dev environment that mimics production, which anyone on the team can then pull down with a simple "docker pull <registry>/<container name>:latest" command.
- falcolas 11y ago
  Docker has its own issues, and in the end is remarkably similar to Vagrant when it comes to development. Docker's biggest win on the development (not deploy) front is that docker containers are faster to set up and tear down; though with a decent Vagrant setup and a package cache, you can build a brand new Vagrant image in under a minute. Since I'm on a Mac, and thus going to be running VirtualBox anyways, I prefer to just work with the VMs directly.
xacaxulu 11y ago
Vagrant is a very well developed tool and has tons of configurations available in the Vagrantfile. It does take a lot of time to get deep into it however. When coupled with Chef's Test Kitchen, vagrant is a powerful tool. Side note; they actively review and accept pull requests.
rev_bird 11y ago
The Vagrant files we've been using have been whittled down to almost nothing -- we use Chef to provision boxes, so the vagrantfile is pretty much just "install Chef, forward ports, run role xxx." At that point, it just pulls the role (and cookbooks) out of the user's local chef repo -- and getting the Vagrant Berkshelf plugin running even removes the need to deal with downloading cookbooks at all.
jtreminio 11y ago
You're not supposed to use Vagrant for the actual provisioning of a box - use the proper tools like Puppet, Chef, Ansible, Salt, etc. And Packer is for generating the base boxes. Packer abstracts away the virtual machine software like Virtualbox, VMWare, Parallels, etc.

davexunit 11y ago

Unfortunately, developing and deploying web applications is still a total nightmare. To build anything useful you'll need about four distinct package managers to get all of the necessary dependencies and some mish-mash of configuration management and provisioning tools to develop it, and perhaps a different set of configuration management and provisioning tools to deploy it. We can do better than this!

deleted 11y ago
[deleted]
danudey 11y ago
Hmm, let's see… > To build anything useful you'll need about four distinct package managers to get all of the necessary dependencies * Ubuntu packages (dpkg via apt) * Python packages (via pip) * Our own software packages (via our own dependency manager) * Custom-built software packages (e.g. nginx) or non-package software installations (e.g. activemq) > some mish-mash of configuration management and provisioning tools to develop it We use salt for (some of) our configuration management along with just symlinks into git repositories (where possible, e.g. MySQL and others won't follow symlinks for security) > perhaps a different set of configuration management and provisioning tools to deploy it We have a custom-built deployment system which we use to deploy git packages (including the configuration packages that the configuration files are symlinked into) as well as our actual software. So… yeah. You pretty much hit the nail on the head.
- davexunit 11y ago
  Thanks for the real life example.

rjbwork 11y ago

Y'know what? Up until Jan last year I had never even touched a server to deploy an app, much less set up a CI/CD/DevOps environment. Now, I'm comfortable with TeamCity, Octopus Deploy, Azure (a ton of hosted services), PowerShell, and a host of other tools. I've set up bunch of HipChat integrations so we know exactly where in our pipleline a particular build is (code pushed, dev build/deploy, and promotions to various environments), if there was a problem with a unit test run, if there were problems with integration tests, and if there are problems in dev/staging/production with our actual live running code via HipChat integrations with our structured log processing server. There are good tools out there and they're pretty usable - just maybe not at your price-point of free for your environment. I went from 0 to 1-click whole-environment deployments in just over a year.

moserware 11y ago
It's nice to hear of other people with a similar stack on HN (Azure, PowerShell, TeamCity, HipChat, and presumably C#/.NET)! It's working well for us too. In addition, we're increasingly doing more with ChatOps with mmbot (as an Azure Web Job): https://github.com/mmbot/mmbot https://github.com/mmbot/mmbot One nice thing about mmbot is that if you use the DLL mode instead of the CSX mode, you get a great IDE debugging experience which makes it even faster to create meaningful ChatOps extensions. I've found it to be a more pleasant development experience that Hubot with CoffeeScript.
- rjbwork 11y ago
  Hah, we also use mmBot, but while it was our original plan to have it be our "DevOps 1 stop shop" we ended up shying away from that idea in favor of a more hands off approach. We still use our mmBot for a lot of back-office querying type operations, but decided that, at least for the moment, TC/Octopus will be our way forward.
mordocai 11y ago
I don't know if you read the same article I did. The author specifically said that we'd come a long way on tools but not made much headway on the people issues related to Dev/Ops. Your comment does not appear to be adding anything to the article besides another anecdote that the tools are good (and if that is what you meant it to be, then I misread the tone of your comment).

Spooky23 11y ago

Of course we haven't. In many cases, DevOps == have developers push code to production platforms. IMO, there's been a lack of attention to the engineering side of the platforms themselves. In the places that I've seen that yammer on about devops, I see a lot of agile projects where version 1.0 works great, but the ongoing maintenance of the platform (whatever it is) is a complete nightmare.

sgoings 11y ago

I think it's worth reflecting that not every improvement initiative (in this case: see DevOps) across the whole world is moving as one large entity. The ending line: "Otherwise in my opinion it feels more like we’re treading water and not making the advancements we could be making." sounds like an existential fallacy to me - it has been my experience that although the "DevOps vision" has not changed significantly, the actual practices and evangelism of DevOps have been infiltrating and transforming the way organizations think about software engineering.

sanderjd 11y ago
Basically my thoughts: 5 years is not a very long time! It isn't treading water, it's taking time to mature and penetrate. It's actually taking root surprisingly quickly, and only looks slow from an early-adopter's perspective. If those talks from 2010 and 2011 were given to substantially the exact same audience as that talk from 2014, then yeah, maybe I see the point that things are just being re-hashed with little progress, but I doubt that's the case. Giving the same exact talks to an ever-expanding audience over 5 years is already a win.

deleted 11y ago

[deleted]

whistlerbrk 11y ago

This recent trend of articles wrt to devops it making me want to post here. Not trying to be too snarky, but why can't people just use shell scripts for automation? They are easy to repair, in a language you (should) know, reproducible, and testable. They are native to your target, supremely portable, etc, etc.

cheeseprocedure 11y ago
Because it's not worth re-implementing the functionality available in Jenkins, Chef, etc. etc. etc.
- stonogo 11y ago
  This is a misunderstanding of the situation. Jenkins specifically has no business being anywhere near a production deployment; it is a development tool, despite "devops" people tending to use it as a web-interfaced crond. Automation should be tooled to the production environment by people who are familiar with the production requirements and -- most importantly -- by the people who will be expected to maintain the environment and respond to critical issues.
  - boon 11y ago
    What's wrong with using jenkins to invoke the local tooling?
  - rev_bird 11y ago
    > Jenkins specifically has no business being anywhere near a production deployment I'm curious about this -- what's wrong with using the same deployment tool for running dev _and_ the other environments? Jenkins itself is a life-saver for, like you said, doing development builds/testing/reporting, but if we've got one script that deploys an image to production, why not trigger it with a Jenkins job, if only to keep everything centralized and maintain coherent deployment histories?
- nickysielicki 11y ago
  Why do you think most people don't use cfengine?

xj9 11y ago

I'm really looking forward to Triton[1] coming out of beta. This may be unpopular/wrong, but I don't think you should have to care about what machine your app is running on in any way. [1]: https://www.joyent.com/developers/triton-faq https://www.joyent.com/developers/triton-faq