9 ms·

DevOps isn't amount making Developers be Ops guys. It's about the fact that automation eats everything, and a significant part of 'ops' is now coding. A DevOps

DevOps isn't amount making Developers be Ops guys. It's about the fact that automation eats everything, and a significant part of 'ops' is now coding. A DevOps person isn't someone who develops, and who does Ops. It's someone who does only Ops, but through Development. It's not about start ups vs Enterprise, it's about 1 person writing programs or 5 people doing things by hand.

mrottenkolber 12y ago
Agreed. And as far as I know, 'fullstack' developer stands for being able to write high level as well as low level code. Hence the word 'stack'.
- thedufer 12y ago
  "fullstack" seems to mean server as well as client these days, since that's such a typical architecture to have. I think the high level+low level thing was the original meaning, though.
tomblomfield 12y ago
Absolutely. One solution is to train up your ops guys in basic Ruby & Chef (for example)
opendais 12y ago
Ya, it would have been better if the title reflected the true target of the rant [Full Stack Developers]. However, I kind of think this reflects a local culture issue and not really a broad one. Like, I'm a full stack developer [e.g. I provision my own prod boxes, write the services that run on them] at my $DAY_JOB. I'm not seeing that as a bad thing unless it gets out of hand and I'm doing that for more than a small cluster of backend services.
aero142 12y ago
This is not the only variant of DevOps. There are companies where the developers are responsible for creating the automation scripts to deploy their code into production as services and expected to keep in running in production.
- nulagrithom 12y ago
  I'm inclined to think they're 'doing it wrong', especially when you cross in to the "keep it running in production" territory.
  - beat 12y ago
    If your production deployments are fundamentally different than your dev deployments, you're doing it wrong. For the most part, you should only be localizing a common deployment pattern. I've suffered through enough three week deployment cycles because the prod environment is almost nothing like dev and everything is done manually. I think I know what "doing it wrong" looks like there.
calinet6 12y ago
DevOps is about improving the systems surrounding the work of software development, so that you (and the whole organization) may produce at a higher level of quality. The OP article is entirely missing the point, and you've set it straight. DevOps and the "Full Stack Developer" are entirely separate problems. DevOps can be specialized as well.
forgottenpass 12y ago
Was (Unix) ops ever not coding? I honestly don't know, I haven't been around that long. But all the old guys I know were "perl is unix duct tape" ops guys. The older, more foundational problems were getting automated back then. Now that they're solved problems, and combined with more and more people running large and/or virtual infrastructure, a new problem domain exists around spinning up machines and deployment. The current coding investment is infrastructure because it's the current pain point. In a decade (or whenever permanent solutions exists for infrastructure) the current way will be considered "by hand" and operations coding efforts will just move onto whatever problem is only visible now that infrastructure is no longer a time sink. You can say that some ops is just admins running already existing software and operating everything by hand, but there will be admins doing exactly that in a decade too.
- zalmoxes 12y ago
  It's about more than just "unix duct tape". It's about 'Infrastructure as Code', treating servers like programming objects. It's about using configuration management tools like Chef and Puppet instead of writing bash scripts which only work on one system.
  - deleted 12y ago
    [deleted]
  - toomuchtodo 12y ago
    "DevOps" here, by which I mean an IT Ops Manager/Linux Admin/Network Admin doing this for more than a decade. Nothing you described is outside of the realm of what your typical linux admin does. I don't have to be a senior python dev to do my job, and I've managed 5500+ virtual machines by myself (puppet/chef, bash, some python, persistent data/object storage). Agree with the author; just shoving more hats onto less people.
- gaius 12y ago
  Yes, the devops movement is silly. In the 90s when I started, every SA knew, and used, Perl and C in their daily jobs. Then dotcom happened and every kid with a Linux box in their bedroom put themselves about as a SA. And in the 10s people think SAs who code is an amazing new invention. And back in the 60s, IBM had "systems programmers"... Same thing.
giulianob 12y ago
"A DevOps person isn't someone who develops, and who does Ops. It's someone who does only Ops, but through Development." That's not true. There are many scenarios where devs also understand ops and do both. It's getting rid of the "throw it over the wall" mentality and incorporating ops within dev itself IMO. And devs love to automate things so we have created a lot of tools for that.
michaelochurch 12y ago
The OP has a point, but his choice of DevOps as the bugbear is clumsy. Maybe the bastardization of it by the business is what he's mad about. "Full stack" is, perhaps, a better target. It's a completely meaningless, useless phrase. The problem is that employers demand specialists, especially for senior positions. At the same time, once they've acquired an employee, they refuse to respect specialties from that point on. Machine learning expert? Sorry, but we need a ScrumDrone over at desk 21-B. Being a software engineer means resolving the fight between your job and your career, which is probably a big part of why this industry is so political. Employers are remarkably inconsistent in this regard. They want sharp people who can interview like real computer scientists, but get in the way of their continuing sharpness (by assigning smart people to dumb work) as soon as they're on board. The insight that the OP has is that employers over-hire for crappy work, and he's completely right, but DevOps didn't do it.
- beat 12y ago
  I think a lot of people in this thread have never actually worked on a Really Big Project. Once you have two or three offshore teams, a hundred developers, associated support staff, multiple product teams, competing customer voices, multiple production environments in different locations with different support, "standards" imposed from external orgs that make no sense for the project at hand... That's when DevOps gets really helpful and valuable. But if you haven't worked in environments like that, you have no idea what it's like.
- dasil003 12y ago
  "Full stack" is useless in the same way that "Agile" is useless. Specifically, it's useless because it was hopelessly cargo-culted and overused due to the original power of the idea. The genesis of full-stack developer is that in the early days of the web you had a long history of programmers, and you had a budding community of web designers and javascript developers homesteading the new medium. For many years, there was an awkward gap in skills in that a good programmer would probably not be able to build a decent HTML/CSS website, and a good DHTML developer or web designer would be completely lost on the server-side. 5-10 years ago a full-stack developer was a very meaningful distinction. Today, every hacker wannabe Uber driver that went to a dev bootcamp for 3 months calls themselves a full-stack developer. "DevOps" avoids this fate only because the subject matter is slightly heavier and harder to fake.
mellery451 12y ago
well, in many places DevOps is implemented as "developers on PagerDuty". When I (the developer) have to be on-call for 7 day rotations, phone by bedside, paged at all hours, then I'm most definitely acting as operations - probably NOT what I signed up for. And, contrary to the stated intentions, I've directly observed developers making crappy, band-aid fixes to ongoing production problems in the interest of "making the pages stop". This is the mindset when you are on call be being paged at all hours. In theory, DevOps is supposed to put those that can best fix things closest to the problems, but in reality a slight separation from the firestorm of ops actually produces better, more thoughtful solutions in the long run. The best balance is to have a first tier Ops on-call, 2nd tier engineering on-call, and any alerting issues get attention within 24 hours, moving to the front of the work-queue. But, indiscriminately assigning everyone "pager-duty" rotations leads to lower quality solutions in the end.
- w_t_payne 12y ago
  Sounds like somebody in the hierarchy doesn't quite "get it".
- dvanduzer 12y ago
  Giving everyone pager duty can lead to higher quality solutions. The band-aid fixes crop up when ownership of a whole system eventually spreads too thinly. Within the right framework, keeping everyone on pager rotation can lead to much smoother operations, because everyone stays familiar with the system as a whole. This was going around recently, and captures the essence of the philosophy: http://catenary.wordpress.com/2011/04/19/naurs-programming-as-theory-building/ http://catenary.wordpress.com/2011/04/19/naurs-programming-a...
  - vacri 12y ago
    In my experience, it also leads to better solutions because devs who don't get woken by issues with their own code are people who don't particularly care about such faults. I've done on-call before where I've begged the devs to fix issues because they were waking me up needlessly. The devs were nice, but somewhat lazy, and my fix wasn't on their radar. Stick them on on-call, and all of a sudden it's more important to fix. At one place I worked we had a two-person support shop. We would claim time and again that this or that affected customers or made support hard. The devs would pick and choose what was fun to work on. I ended up leaving and the other guy went on a prearranged month-long vacation. Everyone else had to pick up support (~5 devs) for a month, and I'm told that they had so much trouble with the normal support load that development actually stopped for that month. Apparently when the other guy got back, they started listening a bit more to his concerns, having had a taste of what happens on the pointy end. In a similar vein, there's a wine distributor where all employees spend their first week half on the phones and half in the packing department, to give everyone a feel of what the core function is and what customers complain about. The guy telling me said that everyone gets the treatment, except the new CEO, who got away with only doing a day rather than a whole week.
copergi 12y ago
The problem is there's as many ideas of what "devops" means as there are people saying "devops". The concept of devops as you just described it makes absolutely no sense to me for example. Operations has always been about automation. CFengine has been around for a long time. So to me, your version of devops just seems like a fad buzzword applied to the same old same old.
- area51org 12y ago
  Better automation than cfengine, with any luck. But in most companies, Operations does not mean automation; that's the very reason the DevOps movement started.
  - copergi 12y ago
    Usually worse, it seems most people use chef or puppet. I understand what you are saying, but I am saying I believe otherwise. Operations has always relied heavily on automation. We managed several thousand unix machines with 3 or 4 people back in the 90s and it was totally normal. Devops started for the same reason cloud started: to put a new buzzword on what people have done forever, so that it can be sold as a new silver bullet.
    - area51org 12y ago
      I'm just not that cynical. We're all tempted to grow cynical after a number of years in this business, but you don't have to let it color your whole world. :-) Yes, we had automation in the '90s — I wrote quite a bit of it myself! — but the landscape has drastically improved. For one thing, the industry is now embracing it, and with the embrace, a name. The name is not being "sold" in any way that I can see — no one is getting rich by bandying around the buzzword. It's not being sold by anyone I'm aware of as some kind of silver bullet, and anyone who believes in an IT panacea deserves what he gets. It is however being used to sell an idea, that automation in the '90s and before was a good thing, and that we should probably do more of it. DevOps means more than just automation, and in large part, these are also improvements in the industry. We're better now, partly because we have to be. For that matter, the cloud is just a name that describes the commodification of computing resources (whether that be actual compute, storage, whatever). Yes, yes, the marketing blowhards of the world have misused and bastardized the word, but that doesn't mean they've ruined it, or that it never meant anything.
ChuckMcM 12y ago
At least in my world view this is a much better definition of DevOps. Folks who make the world run, and through automation can keep a larger portion of the world spinning. It requires someone who can analyze failures, figure out how to predict and mitigate them, and then code automation to do so. Oddly this is much more like the 'developers' of old. If you sat down at a workstation you needed to know how to be your own system administrator, and you needed to write code. Automation has enabled a fairly new class of engineer which I think of as someone who has no idea how the pieces all fit together but they can assemble them with the help of a framework and a toolkit into useful products. They become experts at debugging the toolkit and framework but have little knowledge of how everything else actually works. The problem with this new type of coder is that they can write syntactically correct impossible programs. I didn't understand that until I taught myself VHDL (a hardware description language). VHDL was the first "language" that I knew where you could write syntactically correct "code" which could not be synthesized into hardware. The language expressiveness exceeded the hardware's capabilities (and sometimes you would need to have a time machine). Imagine a computer language where 1/0 was a legitimate statement, not caught by the compiler, but always blew up in your executable. So we have folks who can write code that is grossly inefficient or broken on "real" systems. Google had started a program when I was there to have developers spend time in SRE (their DevOps organization) and it was to invest in them the understanding of what went on in the whole stack so they could write better products. The famous 'times every programmer should know' by Jeff Dean was another such tool. You cannot get too far away from the systems that are going to run your code if you want to write performant code.
- dvanduzer 12y ago
  When Flickr did their DevOps talk in 2009, most of the infrastructure engineers I worked with at the time saw the trend in reverse. The people wearing the Developer hat were relying on our team's ability to automate anything, so the Ops team ended up being the team that best understood how the whole system worked. In 2009, DevOps seemed like there was finally a reasonable answer to Taylorism. Engineers and Programmers and Hardware Technicians and Support Representatives were not cogs in a machine, but humans that could collaborate outside of rigid boundaries. Even at the lowest levels of the support organization, individual workers along the chain could now design their own tiny factories. From there, it's just a matter of communicating tolerances properly up and down the chain. I am probably over-romanticising these notions, but it certainly felt exciting at the time. Not at all like the "fire your IT department" worldview it turned into.
beat 12y ago
Saying "DevOps person" is like saying "Agile person". I think it's a fundamental misunderstanding. DevOps is about getting development and operations to communicate directly and work cooperatively, without impediments beyond inherent complexity.
mjm1138 12y ago
Actually finally created a Hacker News account to come here and say something similar to this. The article is remarkably misguided in several ways, including a fundamental misunderstanding of what "DevOps" means. Being an SA, what rankles me most is the attitude (unfortunately common in the industry) that "As a developer, I could naturally be the world's greatest Sysadmin, if only it weren't such a waste of my amazing talents."
- Jweb_Guru 12y ago
  Yeah. As a developer I used to have a similar attitude, until I saw really good sysadmins (and DBAs) up close. Most developers are kidding themselves if they think they could replace either without at least as much extensive training as it would take one of them to replace the developer.
- MartinCron 12y ago
  I have a somewhat different mindset. "As a developer, I am a demonstrably capable sysadmin, but the organization's discrete silos have made it impossible for me to contribute to ops"
mcormier 12y ago
Sounds like DevOps is new and like Agile isn't always implemented as intended once it gets into the wild at companies. I've experience Agile and Enterprise Agile at companies where meetings are called scrums but are 50 minutes long. Sound like there is DevOps and Enterprise DevOps, or some other bastardization occurring out in the wild. Oh process, oh process, save me from these hardships.
Aqueous 12y ago
Right - and I guess the point is that the person who is working on features ends up also being that one person who does the automatic provisioning and testing pipeline administration work, as well. This is honestly why I've gone with PaaS - mostly Heroku - for several months now when deploying a new application. Why on Earth developers do anything other than working on the core features of their program I don't know. All of the things you need to set up - tesing pipeline, and containerized, automatic deployment, load balancing, databases - are now available as cloud services. There is absolutely no need for the developer to be doing administration and provisioning tasks at this point. If you think you need to set up your own server infrastructure ask yourself one question: is there any specific technical requirement that my application has that can't be fulfilled by existing cloud services? If there isn't, and there probably won't be, you shouldn't be doing ops yourself, especially not in a startup setting where time is absolutely at a premium and you need to be spending all of it on making the best product you can make. And before everyone tells me that PaaS is more expensive - it's only more expensive if your time is worth nothing. But your time isn't worth nothing - it's probably worth over $100/hr if you are a developer working in the United States. So Heroku ends up not being more expensive at all - especially not before you have to scale.
- keypusher 12y ago
  Try convincing the ops at a large bank, insurance company or government that they can run their infrastructure in the public cloud, and watch as you get laughed out of the building.
  - Aqueous 12y ago
    Two, three years ago you'd be right. Rightly or wrongly more and more people are thinking differently about that. The CIA for instance.
  - nickstinemates 12y ago
    I think it depends on your definition of infrastructure. Let me illustrate.. Core banking, batch processing, highly sensitive data stores? Probably not great candidates for public cloud consumption. Web properties and services which don't rely on said functionality? Absolutely great candidates. And the reality is, whether the IT guys like it or not, developers inside of these orgs are consuming cloud because getting a VM in a traditional sense takes forever (for good reason.) As a result, we're seeing a shift in the industry where the idea of large corps / financial institutions / government are being pragmatic about the idea of a 'hybrid cloud.'
yesdocs 12y ago
I can't imagine an environment, other than say a juggernaut company (Google, Amazon, RackSpace, etc) that would require a full-time DevOps engineer. Most companies are delivering a simple service with modest needs. The reality is in these small environments is that their world is fairly small. These small environments rarely go over 50 machines and but a handful of services … so how many times can you automate something? How many new DevOps tasks can be created daily? How many different patterns can a DevOps engineer come across? I’ve done all of these and I quite frankly don’t see a great challenge in this field. In the old days, it was an Architect/Team Lead/Sr Developer who figured out how to distribute a product, then the maintenance and upkeep of the installation scripts was later handed to developers of “less” capabilities. But the architect still reviewed and was keep abreast of installation changes. An Architect/Team Lead/Sr Developer should setup the intial design, scripts etc for use as “DevOps”, but DevOps is not a new engineering discipline. The DevOps tools are trivial to understand (kinda like InstallShield’s VB scripts) and easy to master. However, it does require an engineering discipline. Kids that are used to pushing buttons in NetOps can’t suddenly become an experienced coder … they know little of classical software development. I disagree with the author implying that you should hire a DevOps engineer to do this work so that coders can “code”. The economies of scale are way off, this is something a that a junior developer masters so that he can spend more time coding. I wouldn't recommend companies spend so much on an employee performing a task that is trivial at best. No, this task is a Developer’s job, and as a Developer, you better know about them. I won’t argue that mastering these tools are not time consuming, but if you want to master development, these tools had better be on your roadmap. Software Engineering is a new discipline, what we need to know increases over time and mastering this field is getting harder and harder … kinda like natural selection.
justizin 12y ago
That's completely inaccurate, DevOps is most definitely about getting Developers to think about their software being Operated, and it is not about 'Ops Guys' coding for the first time ever - we always wrote automation tools.
njharman 12y ago
Not really. Devops isn't a role or a person or even a process. It's a way to structure releasing software. It's anti-silo. Instead of group A building software in isolation and then tossing it over wall to group B to deploy. Those groups merge, cross-pollinate or at least communicate frequently. So, the group building software is away of the needs/issues of the group deploying/running the software and vice versa. extend to other groups functions (QA, maintenance, sales). It doesn't matter if one person does a bit of each role or if there are persons for each role. As long as the work closely together.
mlieberman85 12y ago
DevOps means different things to different people it seems. I've spoken with dozens of companies and each one seems to have a different definition. Some view it as an SA with a brain. Others view it as a full stack developer, still others view it as guys focused on developing configuration management and just that. Others consider DevOps tools developers. Some just say DevOps and they mean "guy we can ask to just do whatever needs to get done, helpdesk, networking, configuration management, training, etc"
mikebabineau 12y ago
Strongly disagree. DevOps is not the simple automation of operational infrastructure. That has been done for decades. DevOps is about developer empowerment. It's about creating the systems and tools that give developers more control over the operation of their applications. It's about removing Ops as a technical barrier between new code and live code. At least, (as someone whom most would label as "DevOps") that's what it means to me.
k__ 12y ago
To me it always seemed like DevOps is about Devs who do Ops. Like, I could use an Op for my stuff, but I also could automate "him" away. When I was studying CS, there were a bunch of people, who didn't like to code, so they became Ops. If Ops is now about programming, they can't even resort to this branch of CS...
ldar15 12y ago
From 2009: http://www.rajiv.com/blog/2009/03/17/technology-department/ http://www.rajiv.com/blog/2009/03/17/technology-department/ From 2010: http://dev2ops.org/2010/02/what-is-devops/ http://dev2ops.org/2010/02/what-is-devops/ DevOps is not a department. Its certainly not someone who "only does Ops but through development". Yes, there will be a lot of Ops guys automating things. Yes there will be a lot of "developers" doing deployments (using automation). But please don't start labeling people DevOps. You no more have a DevOps department than you have an Agile department. Its a methodology, not a group.
mh_yam 12y ago
> DevOps isn't amount making Developers be Ops guys Sure seems that way at many places.

mrottenkolber 12y ago

Agreed. And as far as I know, 'fullstack' developer stands for being able to write high level as well as low level code. Hence the word 'stack'.

thedufer 12y ago
"fullstack" seems to mean server as well as client these days, since that's such a typical architecture to have. I think the high level+low level thing was the original meaning, though.

tomblomfield 12y ago

Absolutely. One solution is to train up your ops guys in basic Ruby & Chef (for example)

opendais 12y ago

Ya, it would have been better if the title reflected the true target of the rant [Full Stack Developers]. However, I kind of think this reflects a local culture issue and not really a broad one. Like, I'm a full stack developer [e.g. I provision my own prod boxes, write the services that run on them] at my $DAY_JOB. I'm not seeing that as a bad thing unless it gets out of hand and I'm doing that for more than a small cluster of backend services.

aero142 12y ago

This is not the only variant of DevOps. There are companies where the developers are responsible for creating the automation scripts to deploy their code into production as services and expected to keep in running in production.

nulagrithom 12y ago
I'm inclined to think they're 'doing it wrong', especially when you cross in to the "keep it running in production" territory.
- beat 12y ago
  If your production deployments are fundamentally different than your dev deployments, you're doing it wrong. For the most part, you should only be localizing a common deployment pattern. I've suffered through enough three week deployment cycles because the prod environment is almost nothing like dev and everything is done manually. I think I know what "doing it wrong" looks like there.

calinet6 12y ago

DevOps is about improving the systems surrounding the work of software development, so that you (and the whole organization) may produce at a higher level of quality. The OP article is entirely missing the point, and you've set it straight. DevOps and the "Full Stack Developer" are entirely separate problems. DevOps can be specialized as well.

forgottenpass 12y ago

Was (Unix) ops ever not coding? I honestly don't know, I haven't been around that long. But all the old guys I know were "perl is unix duct tape" ops guys. The older, more foundational problems were getting automated back then. Now that they're solved problems, and combined with more and more people running large and/or virtual infrastructure, a new problem domain exists around spinning up machines and deployment. The current coding investment is infrastructure because it's the current pain point. In a decade (or whenever permanent solutions exists for infrastructure) the current way will be considered "by hand" and operations coding efforts will just move onto whatever problem is only visible now that infrastructure is no longer a time sink. You can say that some ops is just admins running already existing software and operating everything by hand, but there will be admins doing exactly that in a decade too.

zalmoxes 12y ago
It's about more than just "unix duct tape". It's about 'Infrastructure as Code', treating servers like programming objects. It's about using configuration management tools like Chef and Puppet instead of writing bash scripts which only work on one system.
- deleted 12y ago
  [deleted]
- toomuchtodo 12y ago
  "DevOps" here, by which I mean an IT Ops Manager/Linux Admin/Network Admin doing this for more than a decade. Nothing you described is outside of the realm of what your typical linux admin does. I don't have to be a senior python dev to do my job, and I've managed 5500+ virtual machines by myself (puppet/chef, bash, some python, persistent data/object storage). Agree with the author; just shoving more hats onto less people.
gaius 12y ago
Yes, the devops movement is silly. In the 90s when I started, every SA knew, and used, Perl and C in their daily jobs. Then dotcom happened and every kid with a Linux box in their bedroom put themselves about as a SA. And in the 10s people think SAs who code is an amazing new invention. And back in the 60s, IBM had "systems programmers"... Same thing.

giulianob 12y ago

"A DevOps person isn't someone who develops, and who does Ops. It's someone who does only Ops, but through Development." That's not true. There are many scenarios where devs also understand ops and do both. It's getting rid of the "throw it over the wall" mentality and incorporating ops within dev itself IMO. And devs love to automate things so we have created a lot of tools for that.

michaelochurch 12y ago

The OP has a point, but his choice of DevOps as the bugbear is clumsy. Maybe the bastardization of it by the business is what he's mad about. "Full stack" is, perhaps, a better target. It's a completely meaningless, useless phrase. The problem is that employers demand specialists, especially for senior positions. At the same time, once they've acquired an employee, they refuse to respect specialties from that point on. Machine learning expert? Sorry, but we need a ScrumDrone over at desk 21-B. Being a software engineer means resolving the fight between your job and your career, which is probably a big part of why this industry is so political. Employers are remarkably inconsistent in this regard. They want sharp people who can interview like real computer scientists, but get in the way of their continuing sharpness (by assigning smart people to dumb work) as soon as they're on board. The insight that the OP has is that employers over-hire for crappy work, and he's completely right, but DevOps didn't do it.

beat 12y ago
I think a lot of people in this thread have never actually worked on a Really Big Project. Once you have two or three offshore teams, a hundred developers, associated support staff, multiple product teams, competing customer voices, multiple production environments in different locations with different support, "standards" imposed from external orgs that make no sense for the project at hand... That's when DevOps gets really helpful and valuable. But if you haven't worked in environments like that, you have no idea what it's like.
dasil003 12y ago
"Full stack" is useless in the same way that "Agile" is useless. Specifically, it's useless because it was hopelessly cargo-culted and overused due to the original power of the idea. The genesis of full-stack developer is that in the early days of the web you had a long history of programmers, and you had a budding community of web designers and javascript developers homesteading the new medium. For many years, there was an awkward gap in skills in that a good programmer would probably not be able to build a decent HTML/CSS website, and a good DHTML developer or web designer would be completely lost on the server-side. 5-10 years ago a full-stack developer was a very meaningful distinction. Today, every hacker wannabe Uber driver that went to a dev bootcamp for 3 months calls themselves a full-stack developer. "DevOps" avoids this fate only because the subject matter is slightly heavier and harder to fake.

mellery451 12y ago

well, in many places DevOps is implemented as "developers on PagerDuty". When I (the developer) have to be on-call for 7 day rotations, phone by bedside, paged at all hours, then I'm most definitely acting as operations - probably NOT what I signed up for. And, contrary to the stated intentions, I've directly observed developers making crappy, band-aid fixes to ongoing production problems in the interest of "making the pages stop". This is the mindset when you are on call be being paged at all hours. In theory, DevOps is supposed to put those that can best fix things closest to the problems, but in reality a slight separation from the firestorm of ops actually produces better, more thoughtful solutions in the long run. The best balance is to have a first tier Ops on-call, 2nd tier engineering on-call, and any alerting issues get attention within 24 hours, moving to the front of the work-queue. But, indiscriminately assigning everyone "pager-duty" rotations leads to lower quality solutions in the end.

w_t_payne 12y ago
Sounds like somebody in the hierarchy doesn't quite "get it".
dvanduzer 12y ago
Giving everyone pager duty can lead to higher quality solutions. The band-aid fixes crop up when ownership of a whole system eventually spreads too thinly. Within the right framework, keeping everyone on pager rotation can lead to much smoother operations, because everyone stays familiar with the system as a whole. This was going around recently, and captures the essence of the philosophy: http://catenary.wordpress.com/2011/04/19/naurs-programming-as-theory-building/ http://catenary.wordpress.com/2011/04/19/naurs-programming-a...
- vacri 12y ago
  In my experience, it also leads to better solutions because devs who don't get woken by issues with their own code are people who don't particularly care about such faults. I've done on-call before where I've begged the devs to fix issues because they were waking me up needlessly. The devs were nice, but somewhat lazy, and my fix wasn't on their radar. Stick them on on-call, and all of a sudden it's more important to fix. At one place I worked we had a two-person support shop. We would claim time and again that this or that affected customers or made support hard. The devs would pick and choose what was fun to work on. I ended up leaving and the other guy went on a prearranged month-long vacation. Everyone else had to pick up support (~5 devs) for a month, and I'm told that they had so much trouble with the normal support load that development actually stopped for that month. Apparently when the other guy got back, they started listening a bit more to his concerns, having had a taste of what happens on the pointy end. In a similar vein, there's a wine distributor where all employees spend their first week half on the phones and half in the packing department, to give everyone a feel of what the core function is and what customers complain about. The guy telling me said that everyone gets the treatment, except the new CEO, who got away with only doing a day rather than a whole week.

copergi 12y ago

The problem is there's as many ideas of what "devops" means as there are people saying "devops". The concept of devops as you just described it makes absolutely no sense to me for example. Operations has always been about automation. CFengine has been around for a long time. So to me, your version of devops just seems like a fad buzzword applied to the same old same old.

area51org 12y ago
Better automation than cfengine, with any luck. But in most companies, Operations does not mean automation; that's the very reason the DevOps movement started.
- copergi 12y ago
  Usually worse, it seems most people use chef or puppet. I understand what you are saying, but I am saying I believe otherwise. Operations has always relied heavily on automation. We managed several thousand unix machines with 3 or 4 people back in the 90s and it was totally normal. Devops started for the same reason cloud started: to put a new buzzword on what people have done forever, so that it can be sold as a new silver bullet.
  - area51org 12y ago
    I'm just not that cynical. We're all tempted to grow cynical after a number of years in this business, but you don't have to let it color your whole world. :-) Yes, we had automation in the '90s — I wrote quite a bit of it myself! — but the landscape has drastically improved. For one thing, the industry is now embracing it, and with the embrace, a name. The name is not being "sold" in any way that I can see — no one is getting rich by bandying around the buzzword. It's not being sold by anyone I'm aware of as some kind of silver bullet, and anyone who believes in an IT panacea deserves what he gets. It is however being used to sell an idea, that automation in the '90s and before was a good thing, and that we should probably do more of it. DevOps means more than just automation, and in large part, these are also improvements in the industry. We're better now, partly because we have to be. For that matter, the cloud is just a name that describes the commodification of computing resources (whether that be actual compute, storage, whatever). Yes, yes, the marketing blowhards of the world have misused and bastardized the word, but that doesn't mean they've ruined it, or that it never meant anything.

ChuckMcM 12y ago

At least in my world view this is a much better definition of DevOps. Folks who make the world run, and through automation can keep a larger portion of the world spinning. It requires someone who can analyze failures, figure out how to predict and mitigate them, and then code automation to do so. Oddly this is much more like the 'developers' of old. If you sat down at a workstation you needed to know how to be your own system administrator, and you needed to write code. Automation has enabled a fairly new class of engineer which I think of as someone who has no idea how the pieces all fit together but they can assemble them with the help of a framework and a toolkit into useful products. They become experts at debugging the toolkit and framework but have little knowledge of how everything else actually works. The problem with this new type of coder is that they can write syntactically correct impossible programs. I didn't understand that until I taught myself VHDL (a hardware description language). VHDL was the first "language" that I knew where you could write syntactically correct "code" which could not be synthesized into hardware. The language expressiveness exceeded the hardware's capabilities (and sometimes you would need to have a time machine). Imagine a computer language where 1/0 was a legitimate statement, not caught by the compiler, but always blew up in your executable. So we have folks who can write code that is grossly inefficient or broken on "real" systems. Google had started a program when I was there to have developers spend time in SRE (their DevOps organization) and it was to invest in them the understanding of what went on in the whole stack so they could write better products. The famous 'times every programmer should know' by Jeff Dean was another such tool. You cannot get too far away from the systems that are going to run your code if you want to write performant code.

dvanduzer 12y ago
When Flickr did their DevOps talk in 2009, most of the infrastructure engineers I worked with at the time saw the trend in reverse. The people wearing the Developer hat were relying on our team's ability to automate anything, so the Ops team ended up being the team that best understood how the whole system worked. In 2009, DevOps seemed like there was finally a reasonable answer to Taylorism. Engineers and Programmers and Hardware Technicians and Support Representatives were not cogs in a machine, but humans that could collaborate outside of rigid boundaries. Even at the lowest levels of the support organization, individual workers along the chain could now design their own tiny factories. From there, it's just a matter of communicating tolerances properly up and down the chain. I am probably over-romanticising these notions, but it certainly felt exciting at the time. Not at all like the "fire your IT department" worldview it turned into.

beat 12y ago

Saying "DevOps person" is like saying "Agile person". I think it's a fundamental misunderstanding. DevOps is about getting development and operations to communicate directly and work cooperatively, without impediments beyond inherent complexity.

mjm1138 12y ago

Actually finally created a Hacker News account to come here and say something similar to this. The article is remarkably misguided in several ways, including a fundamental misunderstanding of what "DevOps" means. Being an SA, what rankles me most is the attitude (unfortunately common in the industry) that "As a developer, I could naturally be the world's greatest Sysadmin, if only it weren't such a waste of my amazing talents."

Jweb_Guru 12y ago
Yeah. As a developer I used to have a similar attitude, until I saw really good sysadmins (and DBAs) up close. Most developers are kidding themselves if they think they could replace either without at least as much extensive training as it would take one of them to replace the developer.
MartinCron 12y ago
I have a somewhat different mindset. "As a developer, I am a demonstrably capable sysadmin, but the organization's discrete silos have made it impossible for me to contribute to ops"

mcormier 12y ago

Sounds like DevOps is new and like Agile isn't always implemented as intended once it gets into the wild at companies. I've experience Agile and Enterprise Agile at companies where meetings are called scrums but are 50 minutes long. Sound like there is DevOps and Enterprise DevOps, or some other bastardization occurring out in the wild. Oh process, oh process, save me from these hardships.

Aqueous 12y ago

Right - and I guess the point is that the person who is working on features ends up also being that one person who does the automatic provisioning and testing pipeline administration work, as well. This is honestly why I've gone with PaaS - mostly Heroku - for several months now when deploying a new application. Why on Earth developers do anything other than working on the core features of their program I don't know. All of the things you need to set up - tesing pipeline, and containerized, automatic deployment, load balancing, databases - are now available as cloud services. There is absolutely no need for the developer to be doing administration and provisioning tasks at this point. If you think you need to set up your own server infrastructure ask yourself one question: is there any specific technical requirement that my application has that can't be fulfilled by existing cloud services? If there isn't, and there probably won't be, you shouldn't be doing ops yourself, especially not in a startup setting where time is absolutely at a premium and you need to be spending all of it on making the best product you can make. And before everyone tells me that PaaS is more expensive - it's only more expensive if your time is worth nothing. But your time isn't worth nothing - it's probably worth over $100/hr if you are a developer working in the United States. So Heroku ends up not being more expensive at all - especially not before you have to scale.

keypusher 12y ago
Try convincing the ops at a large bank, insurance company or government that they can run their infrastructure in the public cloud, and watch as you get laughed out of the building.
- Aqueous 12y ago
  Two, three years ago you'd be right. Rightly or wrongly more and more people are thinking differently about that. The CIA for instance.
- nickstinemates 12y ago
  I think it depends on your definition of infrastructure. Let me illustrate.. Core banking, batch processing, highly sensitive data stores? Probably not great candidates for public cloud consumption. Web properties and services which don't rely on said functionality? Absolutely great candidates. And the reality is, whether the IT guys like it or not, developers inside of these orgs are consuming cloud because getting a VM in a traditional sense takes forever (for good reason.) As a result, we're seeing a shift in the industry where the idea of large corps / financial institutions / government are being pragmatic about the idea of a 'hybrid cloud.'

yesdocs 12y ago

I can't imagine an environment, other than say a juggernaut company (Google, Amazon, RackSpace, etc) that would require a full-time DevOps engineer. Most companies are delivering a simple service with modest needs. The reality is in these small environments is that their world is fairly small. These small environments rarely go over 50 machines and but a handful of services … so how many times can you automate something? How many new DevOps tasks can be created daily? How many different patterns can a DevOps engineer come across? I’ve done all of these and I quite frankly don’t see a great challenge in this field. In the old days, it was an Architect/Team Lead/Sr Developer who figured out how to distribute a product, then the maintenance and upkeep of the installation scripts was later handed to developers of “less” capabilities. But the architect still reviewed and was keep abreast of installation changes. An Architect/Team Lead/Sr Developer should setup the intial design, scripts etc for use as “DevOps”, but DevOps is not a new engineering discipline. The DevOps tools are trivial to understand (kinda like InstallShield’s VB scripts) and easy to master. However, it does require an engineering discipline. Kids that are used to pushing buttons in NetOps can’t suddenly become an experienced coder … they know little of classical software development. I disagree with the author implying that you should hire a DevOps engineer to do this work so that coders can “code”. The economies of scale are way off, this is something a that a junior developer masters so that he can spend more time coding. I wouldn't recommend companies spend so much on an employee performing a task that is trivial at best. No, this task is a Developer’s job, and as a Developer, you better know about them. I won’t argue that mastering these tools are not time consuming, but if you want to master development, these tools had better be on your roadmap. Software Engineering is a new discipline, what we need to know increases over time and mastering this field is getting harder and harder … kinda like natural selection.

justizin 12y ago

That's completely inaccurate, DevOps is most definitely about getting Developers to think about their software being Operated, and it is not about 'Ops Guys' coding for the first time ever - we always wrote automation tools.

njharman 12y ago

Not really. Devops isn't a role or a person or even a process. It's a way to structure releasing software. It's anti-silo. Instead of group A building software in isolation and then tossing it over wall to group B to deploy. Those groups merge, cross-pollinate or at least communicate frequently. So, the group building software is away of the needs/issues of the group deploying/running the software and vice versa. extend to other groups functions (QA, maintenance, sales). It doesn't matter if one person does a bit of each role or if there are persons for each role. As long as the work closely together.

mlieberman85 12y ago

DevOps means different things to different people it seems. I've spoken with dozens of companies and each one seems to have a different definition. Some view it as an SA with a brain. Others view it as a full stack developer, still others view it as guys focused on developing configuration management and just that. Others consider DevOps tools developers. Some just say DevOps and they mean "guy we can ask to just do whatever needs to get done, helpdesk, networking, configuration management, training, etc"

mikebabineau 12y ago

Strongly disagree. DevOps is not the simple automation of operational infrastructure. That has been done for decades. DevOps is about developer empowerment. It's about creating the systems and tools that give developers more control over the operation of their applications. It's about removing Ops as a technical barrier between new code and live code. At least, (as someone whom most would label as "DevOps") that's what it means to me.

k__ 12y ago

To me it always seemed like DevOps is about Devs who do Ops. Like, I could use an Op for my stuff, but I also could automate "him" away. When I was studying CS, there were a bunch of people, who didn't like to code, so they became Ops. If Ops is now about programming, they can't even resort to this branch of CS...

ldar15 12y ago

From 2009: http://www.rajiv.com/blog/2009/03/17/technology-department/ http://www.rajiv.com/blog/2009/03/17/technology-department/ From 2010: http://dev2ops.org/2010/02/what-is-devops/ http://dev2ops.org/2010/02/what-is-devops/ DevOps is not a department. Its certainly not someone who "only does Ops but through development". Yes, there will be a lot of Ops guys automating things. Yes there will be a lot of "developers" doing deployments (using automation). But please don't start labeling people DevOps. You no more have a DevOps department than you have an Agile department. Its a methodology, not a group.

mh_yam 12y ago

> DevOps isn't amount making Developers be Ops guys Sure seems that way at many places.