20 ms·

DevOps: An idea so good, no one admits they don’t do it

235 points by wagslane 4y ago

t_mann 4y ago
If you have a separate DevOps role in your org, arguably you're not doing DevOps. So probably most orgs aren't doing DevOps, at least those that are talking about it.
draw_down 4y ago
I see the point, but it’s funny that the author lays out the reason why this goalpost-moving is happening. As pointed out, you sound like an idiot if you’re against Agile. So it’s a lot easier to get what you want if you just call it Agile. I get that it’s not intellectually rigorous or whatever, but good grief, when has that ever mattered in business. Is there any class of concepts that survives unblemished after contact with reality?
- tuckerman 4y ago
  I'm not sure I agree with the author wrt Agile. I've never been on a team that did anything that resembled agile but I have been on teams that actively pushed back when anything that looked like it sort of resembled agile got floated. I'm not saying its 100% bad but I'm not sure it has a universally good reputation
  - draw_down 4y ago
- rufflez 4y ago
  Agile is bogus nonsense - there I said it.
  - latchkey 4y ago
    Cool... what's your alternative?
    - rufflez 4y ago
      Software used to be built before Agile... Let's at least keep solution design and also get senior engineers to help estimate And do away with daily stand up and bs rituals.. a meeting is needed when it is needed, not for the sake of "following the process"
  - kenrose 4y ago
    Yeah, iterate quickly, learn from your customers usage, repeat. You're right, total rubbish. /s
    - stonogo 4y ago
      Your pithy dismissal leaves out hundreds of pages of religious rites and project management dances which invariably turn into productivity-sucking meeting overload, unachievable goals, and constant product churn, effectively alienating everyone from the developer to the user, leaving only the managers satisfied. Like telling a college student that communism doesn't work, any objection is met with "well they're just not doing REAL Agile," skipping over the fact that nobody on Earth seems to be "doing REAL Agile." Agile could just as well serve in the headline in place of DevOps.
    - rufflez 4y ago
      This is not how it works in real life. Contracts are made to the lowest bidder, there are promises to keep or heads will roll. By iteration 10, your customer is fuming that the half baked idea has more exceptions than handled scenarios and the finger pointing begins. Also, forget about architecture, performance, security and other pesky little problems. I have never seen it play out differently. The biggest problem is that the person footing the bill does not care about your story points or retrospective- and I don't blame them for that
- drewcoo 4y ago
  > Is there any class of concepts that survives unblemished after contact with reality? We're not talking about reality, though. We're talking about things consultants market to management.
itsmemattchung 4y ago
I was one of the biggest proponents of DevOps (even started and the DevOps orange county meetup) but ... after working at AWS for 5+ years, my entire perspective shifted. While DevOps does work in some organizations, sometimes what you really need to do is operationalize your software development team. Don't get me wrong, system developers/engineers/operations are needed but for sometimes for a small four person start up, the best investment is improving the development team's operational posture (e.g. run books, alarming, monitoring).
- 0x457 4y ago
  > for a small four person start up, the best investment is improving the development team's operational posture (e.g. run books, alarming, monitoring). But that's literally what DevOps is? DevOps isn't a special DevOps team that writes terraform.
- calvinmorrison 4y ago
  > development team's operational posture (e.g. run books, alarming, monitoring. Isn't DevOps in large part forcing the developers to have responsibility over the code they push and giving short and frequent pushes to easily rectify problems? Making developers do support work reduces support in the long run because they'll get frustrated and automate solutions.
  - stetrain 4y ago
    Yeah, I feel like the term has been taken over to mean using kubernetes and having continuous deployment pipelines for your microservices. My understanding of root of the term is about ownership and responsibility, where the team that develops the app also owns how the app deploys and runs in production, including monitoring and responding to issues. If the team decides that the right way to deploy their app is once every two weeks using a shell script and a zip file, cool. As long as it's documented and monitored so you know if it's working.
    - calvinmorrison 4y ago
ciguy 4y ago
DevOps has lost all meaning as the first paragraph in the article says. Anything it says after that is just semantics. Everyone is doing it on some level, but what that means to each company differs widely.
mnd999 4y ago
All these processes are a template, a starting point. Begin there and optimise and refine to suit your team and your environment. Dogmatically following a process that doesn’t work for you is silly.
paxys 4y ago
I worked at a large internet company in the mid-late 00s, and the typical team composition was a handful of developers, a product manager, a designer (if you worked on customer facing UI), a project manager (for large enough projects), QAs, and a SRE. Every team had dedicated engineers whose sole purpose was to place purchase orders for servers, set up and manage databases, deploy changes, monitor uptime and more. And this was standard practice across the industry, as soon as your company outgrew a shared host. Even the thought of something like this would be absurd today. So I'd flip the "no one admits they don't to it" around – your company does DevOps already, whether you realize/admit it or not.
- dilyevsky 4y ago
  Replace “purchase orders for seevers” with “apply terraform to spin up VMs” and ditch the QA person and not much has changed
  - deleted 4y ago
    [deleted]
- water-your-self 4y ago
  In the mid 2000s was the role referred to SRE yet? Facebook calls it production engineer, DevOps is broad/ im unsure its origin, Google calls it SRE.
  - jbverschoor 4y ago
    Within non-tech companies, they call everything "the programmer"
- jzawodn 4y ago
  Yahoo?
ipnon 4y ago
SRE seems the better approach. Make an error budget, measure reliability, stop releasing in favor of reliability engineering if the budget is missed, use the free time reduce toil. DevOps seems to degrade to disorganization because the incentives are not aligned between product development and software engineering.
- jameshart 4y ago
  “Make an error budget, measure reliability, stop releasing in favor of reliability engineering if the budget is missed” These are all things that fall in the realm of ‘ops’. ‘Dev’ops is an approach to ‘doing ops’. It is one that adopts certain practices that are common in ‘dev’elopment - such as defining things programmatically, using automated repeatable processes to generate work product, and using automated testing and verification. So what you describe as ‘SRE’ doesn’t seem to me to be ‘not devops’. Seems like a set of ideas that would benefit from a devops kind of approach.
- tootie 4y ago
  I think you've hit at the heart of the problem, but swapping the DevOps model for SRE is robbing Peter to pay Paul. Last place I worked with a 100+ person tech team had more of an SRE model. The SRE team were charged with reliability. They had no incentive to delivery features or to improve developer experience. As a result, developer tools sucked, there was s rigid release schedule, every bug was met with a rollback. This was probably an extreme case because the team was also not very good, but either way it was a clear case of misaligned incentives. The entire point of DevOps was collective ownership with a shared goal of customer value that is function of feature richness, quality and reliability. It's the Holy Grail of efficiency, it's just really hard to build that kind of culture.
  - ipnon 4y ago
    Seems like the sweet spot is DevOps style reliability ownership with SRE style indicators, objectives and agreements.
- jedberg 4y ago
  If you think SRE and DevOps are opposed to each other, then you might be doing one of them wrong. I headed up SRE at Netflix, and our sister teams wrote the software and managed the tools that enabled the developers to continuously deploy and operate their own services. We were in charge of reliability, but we did it through chaos testing and accountability, and enhancing the deployment tools to warn devs that they might be doing something high risk, but still letting them do it.
drewcoo 4y ago
The entire article avoided talking about definitions of DevOps, while rambling a lot about how we have different ideas of what it means and how we need better definitions. I appreciate the alert that there's a problem here, but there's no substance to the article.
vesinisa 4y ago
The author is right that DevOps seems to often mean different things to different people. I would have loved it if they had taken more of an attempt at one definition than just that it's when developers have an understanding of how and where their code is deployed. That does sound sensible, but what other best practices does DevOps usually entail?
giantg2 4y ago
I feel like DevOps is a trap. Most companies that have sufficient talent and organization/operational setup don't see a need for DevOps. There seem to be a lot of other companies that are sucked in by its promises and think that it will solve their issues. Surprise... this didn't fix your shitty policies, disorganized/unempowered org structure, or cheap talent (cause you're unwilling to pay or train). I assume there are a few places that can benefit from this. Maybe some medium to large tech companies with great procedures, documentation, and an empowered org structure. Especially if they're paying enough for people who can manage it. The other is probably for very small places wirh fairly simple tech where you can't afford entire teams of people so you empower someone to really own the system top to bottom (without ever calling it DevOps).
- turtlebits 4y ago
  IME, letting developers have free reign over service architecture/deployment/CI/observability is a bad idea. Especially when every team comes up with their own option. You need some sort of Devops (best practices if not a team) to provide guidance and help implement standards for these.
  - giantg2 4y ago
    I agree, if we're talking about places big enough to have multiple teams. The thing I was pointing out is that most places that have multiple teams are terrible at setting up best practices. They'd be better off with a centralized "environments" team provisioning the server, network, etc so the dev teams can work on the actually implementation. At least it's easier to maintain standards, or at least consistency, across apps if the low/mid level config is handled by one team. When I was talking about small places, I mean like tiny mom and pop non-tech companies with an IT department of 1-5 people.
- ryukoposting 4y ago
  Every company that has a software product also has DevOps, whether or not they call it that. If someone manages your cloud infrastructure, they're doing devops. If someone is coming up with CI/CD setups, they're doing devops. If someone came up with SOP for distributing new releases, they're doing devops. Whether or not anyone's title has "DevOps" in it, devops tasks are still devops tasks. By the same logic, even though I watered the office plants this morning, I am still a firmware engineer- not PlantOps. The real trap of DevOps is the assumption that it's anything new or non-essential. DevOps has existed since the notion of "software as a product" came into existence, we just didn't have a name for it yet. None of this is to say you need a dedicated DevOps person (or PlantOps, for that matter). DevOps is an integral part of any software product, even if no specific member of the development team is dedicated solely to DevOps tasks.
ebiester 4y ago
Or, alternatively, it might not be a great idea in all cases. Maybe your development team shouldn't develop terraform the four times a year they need it. It might even be faster to have two or three individuals with a singular vision for the system in a smaller team. (There may not be enough work for some teams to have someone with specific DevOps skills. However, that ends up in a partial-shared-service model where team members get called off for organizational concerns.) Maybe you need a subject matter expert that will develop standards for teams and select a set of tools, setting those up so that teams can build off those for their needs. (This is why Ops are often shared services teams.) Maybe it's not a great idea if the first team to encounter a problem picks the tool that works best for them, and the organization ends up with multiple tools because there was nobody with both the time and expertise dedicated to the analysis. (This is why Ops are often shared services teams.) We need to break down barriers. We need to adopt valuable practices. However, the parts of DevOps that don't make it to the real world may just be because they don't solve the problems of the organization.
JohnFen 4y ago
As a dev, the whole "rapid release" thing made a lot of sense to me. As a user, though, I've turned 180 degrees on it. I hate "rapid release" with a passion now. Which means I dislike Agile by consequence. Rapid release means I can no longer rely on the software that engages in it. It's constantly shifting, with frequent changes in UI and workflow, and new and exciting bugs constantly appearing. I miss the days when I could plan ahead for new versions, so I could time them to have the least possible disruption to my life.
- cjsawyer 4y ago
  I miss being able to actually refuse an update. Or even being able to disable them!
  - dleslie 4y ago
    I miss knowing that updates even happened. I also miss important features remaining stable. Which is why, for the most part, I've stopped using web-based applications.
    - ethbr0 4y ago
      You want a chuckle, turn off automatic app updates on an Android phone and see how many prompts you get. And how terrible the patch notes are ("Fixed some things")
      - dleslie 4y ago
        In fact, my Android 11 device has no browser enabled, the play store is disabled, and almost every app it came with is disabled. It functions solely for email, instant messaging, a TODO tracker, and taking photos. Edit: not sure why this was downvoted. Did someone find the idea of a narrowly-focused phone to be offensive or unworthy of discussion?
        xwolfi 4y ago
        We know but OP was telling you to experiment: turn off auto update and turn on the play store, and see how many prompts you get. We're not sure why you went to describe how you read emails, and that's why we downvote you. Your answer was completely irrelevant !
the_jeremy 4y ago
My team at Nasdaq (I work on other products, not the stock exchange) does DevOps surprisingly well for not being a Real Tech Company TM. I don't know that the business sees benefits, as it meant that all of the devs on my team have had to learn Terraform, Helm, AWS, and Kubernetes in addition to the actual SpringBoot Java application code, so our feature velocity dropped quite a bit, but it was fun to learn, and it's probably better to have people who understand the full deployment and not just pass blame between teams during incidents.
- lmarcos 4y ago
  Oh man, that's the trap. I do like to learn new things, I swear. But at work I want to focus on what I do best: feature development. Sure, I do know terraform and k8s and what not (because I play with them via my side projects), but at work don't give me the Ops/Platform hat please! It's cheap labor (your company doesn't want to hire a dedicated platform engineer to handle terraform/k8s/etc. and instead assigns you (the developer) that responsibility claiming "you are gonna have fun learning" and paying you 25% more in the best scenario)
  - MetaWhirledPeas 4y ago
    I don't know if you're right or not, but I do think your hesitancy mirrors why a lot of organizations are unable to adopt DevOps fully: many people don't want to wear a bunch of hats. They just want to do their one thing, and I can't really blame them.
    - tremon 4y ago
      But I don't think the original DevOps (TM) specified that every person was required to work on every part of the stack? Just that you need to have full-stack skills and responsibilities within the same team? That said, I probably wouldn't want to hire someone with just one hat. It's ok to have a preferential hat, it's not ok to refuse to wear any other.
      - BlargMcLarg 4y ago
mkl95 4y ago
Most SaaS are overcooked spaghetti floating on AWS alphabet soup. A devops mentality won't fix them.
robertlagrant 4y ago
> If I were to generalize, I’d say most of the companies I’m picking on fall into one of two categories: > They believe in DevOps ideas, but they’re unwilling to invest in changing their ways. They aren’t convinced that some of the scarier ideas (like deploying multiple times per day) will help after all. Option number 3 is the biggest one for me: thinking that having a separate "DevOps" team from the Dev team that does ops via modern tools is DevOps. That may be a good approach for your company, but it ain't DevOps.
spiralpolitik 4y ago
DevOps (and Agile) both usually run aground on the same question of trust. If you aren't willing to trust your delivery teams then you aren't going to see the leverage that both practices bring to an organization. And if you aren't willing to trust your delivery teams then your organization has a much bigger problem than either DevOps or Agile can solve.
tibbon 4y ago
I also go to the gym 4 times a week, go to sleep on a regular schedule and eat a balanced diet!
alexjplant 4y ago
According to "The DevOps Handbook" DevOps is... > the outcome of applying the most trusted principles from the domain of physical manufacturing and leadership to the IT value stream. DevOps relies on bodies of knowledge from Lean, Theory of Constraints, the Toyota Production System, resilience engineering, learning organizations, safety culture, human factors, and many others. Anybody that's worked in an organization that "uses DevOps" or has "DevOps Engineers" knows that this simply isn't true in the vast majority of circumstances. Too often DevOps is just a new name for an ops team. "True" DevOps is all about aligning incentives, sharing responsibilities, and fostering collaboration between people who build and maintain systems whereas "DevOps" (with air quotes) is just a new spin on being a sysadmin with a greater emphasis on AWS and Terraform.
- baobabKoodaa 4y ago
  Both the quote and your response to it read like word soup to me. I have literally no idea what the point in either of these are. Historically, the word "DevOps" meant that instead of having separate "Dev" and "Ops" teams, the same people would do both Dev and Ops work.
  - lazide 4y ago
    Which is what they’re saying. Because they do both jobs, folks understand both sides, and work together to make things work better end to end for them and for customers. Alternatives include siloed dev teams that throw shitty software that crashes in production all the time to an overworked ops team that takes all the blame. And overworked ops teams that have gone pager-blind and only care if a VP yells at them, so ignore all but the most major issues - after they already happened.
    - 0xbadcafebee 4y ago
      The fact that engineers are not taught those topics are why DevOps has failed. We're expecting whole organizations to magically improve when they don't understand how. I can count on one hand the number of people I have met who understand OP's comment.
rhacker 4y ago
No one admits they don't do it because everyone does it. I have worked at 6 different companies since I've heard that term and all of them were able to deploy to an environment by clicking a button on a CI/CD tool. Maybe I'm the outlier?
- 0xbadcafebee 4y ago
  Anyone can deploy a file with one button. DevOps is what happens when the app starts randomly crashing. If the site keeps humming along like normal, or the app is automatically rolled back, or (rarely) the devs get a page & quickly diagnose in production and apply a fix in 10 minutes, and deploys don't randomly fail due to unpinned versions etc, that's DevOps. If people start running around like chickens with heads cut off and pointing fingers, that's not DevOps.
  - leetrout 4y ago
    I like this distinction. I think it succinctly captures a big difference.
vinaypai 4y ago
Not directly related to the question, but I personally hate the term "Sprint". Physiologically, a sprint involve by definition is a short-term high effort activity that can't be sustained without a break. Does anyone use a different term that doesn't imply working oneself to death?
- duped 4y ago
  Cycle
- 0xbadcafebee 4y ago
  Just changing the word isn't going to change the practice. You're supposed to take breaks between sprints. There needs to be planning and alignment to an easily-obtained goal, which requires time. But most people don't know how to do Scrum and hand-waive the goal, the planning, the retro. The end result is unsustainable work with no specific goal in mind that exceeds the sprint cycle. Or 20 epics all in progress at once. Or business goals that shift and put unobtainable expectations on teams.
  - bot41 4y ago
    What's a break period? Where is that described?
    - 0xbadcafebee 4y ago
      It's not explicitly described. "The break" (imo) is giving enough time to planning at the beginning, and actually dealing with the review & retro items. The length of the sprint can be shortened so that there is more learning and planning. Sprints are cancelled if the goal becomes obsolete.
- 0x457 4y ago
  Cycle or stint. I like "stint" more.
wiremine 4y ago
Recently I've been reading "When Agile Gets Physical" [1] and "Extreme Programming Explained": 1. "Extreme Programming Explained" explains the importance of separating values from principles and practices. 2. "When Agile Gets Physical" unpacks why using agile software best practices are horrible for hardware. This is "duh" if you're a hardware person, but the book goes back to agile/lean first principles, and then builds up an approach for agile hardware. It got me thinking that we often conflate values and practices. Blinding adopting Scrum or DevOps without understanding some of the underlying value and principles leads to problems. If you understand the foundational principles, you're in a much better position to adopt/reject certain practices for your given situation. In reality, most organizations just adopt a few agile practices, while rejecting the basic values. [1] https://rapidlearningcycles.com/when-agile-gets-physical/ https://rapidlearningcycles.com/when-agile-gets-physical/ [2] https://www.amazon.com/Extreme-Programming-Explained-Embrace-Change/dp/0321278658 https://www.amazon.com/Extreme-Programming-Explained-Embrace...
dec0dedab0de 4y ago
I was a network engineer at my previous job, I was fed up with doing things manually so I learned to code better. Now I'm a full stack developer writing code for the network team of a different company. One thing that has changed drastically over the last 15 years is the sharp divide between IT and development. It used to be that developers were clueless how to deploy their own apps, and IT was terrified of writing code preferring to do things manually, or purchase something prepackaged. Now I see developers that understand firewalls and load balancers, and IT folk that can script stuff out for themselves. I think we have the push behind DevOps to thank for that progress, and while there is still a long ways to go, I think it's a good thing overall.
- mxuribe 4y ago
  > ...writing code for the network team of a different company... Something like is the kind of job that i think i want to do next. It need not be for a network team...but i'm headed more for back-end work and these days i rather engage with internal teams. Separately, to your comment on sharp divide...The sharp divide that i have seen is a lack of cohesion for the following 4 IT groups: first-level support which gets sent to off-shore; the folks who (honestly have the most face time with internal clients who) roam around the office fixing desktops and building network/wiring, and setting up/fixing A/V equipment in conference rooms; sys admins; and developers. Furthermore, in the last 2 companies that i worked for, there has been a push to "centralize" certain functions like IT and HR to "shared services"...but at the end, none of these groups seem centralized! They all now work more separated than ever before...and most of the interaction between each of these groups - and even between the 4 subgroups i noted above in IT alone - mostly communicate via tickets, or some other thick-layered mechanism. Sure, they've been centralized in that these people were pulled from some respective business unit, and brought under a more single umbrella...but in the end, these workers hate it, the effecicny drops, more bureacracy is created (which creates no value)...and some executive gets to lead the shared services...and of course some management consultant company gets their payday. Oh, and yeah, IT at these orgs definitely does not use any of the good parts of devops, agilescrum whatever. Some orgs try and adopt some cookiecutter approach only to have it create yet more bureacracy...but "hey, we're agile!" /sigh
jbverschoor 4y ago
Devops means let a non-admin developer have access to aws and just buy everything he can find in order to try and fix his shortcomings. It'd be better and cheaper to follow a sysop course
nkotov 4y ago
The mistake was turning DevOps into a job position which resulted in silos being created more when the whole purpose was to remove/prevent silos and to allow developers to operate at their own pace.
honkycat 4y ago
"DevOps" is a corrupted term. It just means sysadmin at this point.
- mikkergp 4y ago
  While I get the underlying argument you are trying to make, there are definitely still sysadmins in the world who see their job as administering systems, and refuse to look at anything resembling automation.
- bot41 4y ago
  I think it means devs that write their pipelines themselves. Someone else has to enable the devs to do that
synu 4y ago
I feel like devops became a marketing juggernaut then somehow evolved into creating a third team, apart from dev and ops, that does handoffs with both of them.
jascii 4y ago
I think one of the problems with the terms "Agile" and "DevOps" is that they are pretty general and generic terms that they are trying to cast very specific meaning too... “So you’re saying you’re not an agile team?” Sure we are appropriately agile for our needs, no need to be rigid about it...
notabee 4y ago
Corporate kayfabe. https://en.wikipedia.org/wiki/Kayfabe https://en.wikipedia.org/wiki/Kayfabe
nont 4y ago
This is also very true in the presales engagements by top consulting firms under "Digital Transformation". These presales teams would come and show, on paper, how great they are with the agile practices, devops, microservices, you name it. However, it's a totally opposite story during the actual implementations. I've witnessed that it took a top consulting firm 4 months to provision a sandbox environment while being 1 month away from the first release. When asked about their proud DevOps practices, you'd hear excuses that are in the blatant lie territory. They knew they could get by by managing the stakeholders instead of really doing what they had promised.
0xbadcafebee 4y ago
I agree the terms have become pointless. I have been thinking about opening an actual school for DevOps, or at least IT Ops. There is virtually no education for it, but it's a complex subject matter that requires years of experience. The SRE books are a peek under the covers but leave so much unsaid that it's really only useful at "doing operations", not digital transformation. To properly build and run a high-performing organization, you need to understand at least 1) the entire history (up to today) of software development practice and production operations, 2) all of the competing requirements of product design, development, and operation, 3) architectural best practices, 4) the industry standards that apply to the work, and 5) tech product-aligned business management methods. Basically you need a generalist who has 15+ years experience and has tried and failed at everything under the sun. I think of it like carpentry before there was a building code, and before the introduction of apprentices and licenses. Trying to build high-performance orgs without a master builder to show you is like trying to build skyscrapers after having built your own home (and having never seen a skyscraper). Or worse, taking an old barn (non-tech enterprise) and trying to build a skyscraper out of that. Without a trade school it will continue to be just a craft.
benreesman 4y ago
When I was a wee lad, “programmers” wrote software and sorta tested it and then kicked it over the fence to “systems administrators”, who chain-smoked and were very, very un-woke. This was a bad system. To the extent that “DevOps” means anything, it’s that the spheres of capability and responsibility need to have tons of overlap: SWE’s need to be in the oncall loop and know how the filesystem works, and SRE’s need to be serious coders. That’s it: everyone codes, everyone operates. We all automate where feasible. Sidebar: What’s with the crazy surge in BuzzFeed-style titles on the front page? It’s not a federal matter but it’s not a slight change YoY either.
- kazinator 4y ago
  The test cases for Unix must have been kept on some tapes that were lost, because a test case is hardly to be seen in the historic sources you can find online.
oxfordmale 4y ago
Agile is a philosophy, it is not sprints, story points or whatever crazy methodology an expensive Agile Coaching company has come up with. The same applies to DevOps. DevOps could just be a bash script, written by the developer, to feit their codebase. It doesn't have to be fancy or require an extensive amount of investment.
- mxuribe 4y ago
  You sound like Mr. Miyagi from the Karate Kid movies (which is a good thing!)...specifically it remains me of that scene when Daniel-San asks him what "belt" he is...and Mr. Miyagi responds "Calvin Klein"! (At some point, Mr. Miyagi explains that his exppertise of Karate is only for the need to survive encounters, and not worry about which "belt" he had achieved.)
irrational 4y ago
I hate devops. Mainly because I don’t want to do it. I liked it better when there was a dedicated infrastructure team instead of pushing it down to developers and giving it a new name.
- hedora 4y ago
  Yeah. As a developer, I need a competent QA and operations team to support me. I'm not competent at those things, but 90% of developers I've worked with are somehow worse at them. I also need 1-2 months between major context switches, and the ability to declare a subcomponent "ready for release". Moving forward, I will actively avoid companies that claim to use agile, scrum or devops. Edit: Also, I want to work on a team that has a good release manager.
MetaWhirledPeas 4y ago
> We need better definitions for these terms How about this? Let's describe levels of adoption. Think of the "normal form" concept for databases. Each level has a strict definition, and you can objectively observe which level you are using. And not every level is appropriate at all times. So maybe a team could say, "we're at DevOps level 2," or, "we use level 3 Scrum."
- leetrout 4y ago
  Yes. I like this concept a lot. Maybe accompanied with a landscape of verticals and how you progress in each one.
AtNightWeCode 4y ago
You go up at 5 AM in the morning. You ride your bike to a customer. You pass some security checks like, known guy, yes, no. You are shown to the servers in the cellar. Cause they are safe there. You get access to the admin account after 2 hours. You do some backup to a local disk. Somebody screams something but you can’t hear what because of the noise. You say yes and hopes it was about coffee even though the room is like a sauna. Now you pick up that CD and insert it. You are not allowed to use a USB-drive because it is mutable. Some clicks and the installation begin. And now you wait. I am pro-devops.
nimbius 4y ago
when John Willis explained it in 2016 devops had enormous potential for the initiated. id guess maybe 10% of anyone who says 'devops' does anything remotely close though. devops in 2022 means if youre a dev, youre expected to know the bowels of the OS inside and out as well and be willing to spend hours debugging it alongside your normal programming work. if youre in ops? it means debugging someones undocumented middleware from nine years ago while the front of site burns so "rockstar" coders dont jump ship having to do it instead. if youre in dev or ops, it now means standups where you sit down for an hour and talk about where to put your story in jira or how to enter it at all, and where you discuss broad strategy too. instead of leaving after your status update, youre expected to stay. its just another meeting now. and once boardmembers and leadership realized they could spare themselves the crucifixion of any real accountability after a breech, they turned devops into devSECops, meaning your infosec team sits in awkward silence as you explain why double defined nginx headers arent a feature and the same management team that made them part of the standups also unilaterally declared ssl3 can never be deprecated because customers.
- encryptluks2 4y ago
  This is my issue. Companies have commoditized the average developer and now treats it like a average entry level role
jammycakes 4y ago
In a previous job, I joined a team that was supposed to be introducing DevOps to the organisation. It started out well -- we spent a few months hacking with Terraform, Docker, Vagrant, Kubernetes, and related technologies to implement an infrastructure-as-code approach -- automating the process of provisioning and decommissioning servers, and building a setup where development teams could deploy updates with a simple git push. Unfortunately it all went downhill fairly rapidly. We ended up spending the majority of our time manually applying security patches to a bunch of snowflake servers that we'd lifted-and-shifted from another hosting provider to AWS, and fielding support requests from the development teams. Within a year, we were being told in no uncertain terms by our project manager that we were an operations team, not a development team. It felt like a complete bait-and-switch. Within two years, I had left the organisation in question and moved on to a new job elsewhere doing actual development again. Last I heard, the entire team had been disbanded. It sounds like the author of this article must have had a very similar experience. I wonder just how common it is. It seems that in many places, "DevOps" is all Ops and no Dev.
- spaceywilly 4y ago
  > We ended up spending the majority of our time … fielding support requests from the development teams This has been my experience at 3 different small-medium companies now. A too small DevOps team suddenly is in the critical path for even the most trivial software task, then engineering productivity grinds to a halt. I think a much better pattern would be to enable dev teams to self serve. Set up the required infrastructure and guard rails, then let teams handle their own deployments and infrastructure. Give people what they need to do it themselves instead of having to open a support ticket for everything.
  - lazide 4y ago
    Most developers won’t know how to do it without footgunning themselves constantly is the problem. If the dev ops team is staffed enough to develop integrations that won’t allow that AND won’t get in the way, and then train folks and ‘keep the line’ enough to stop the scope creep - they’re probably not at a shitty small/medium sized shop.
g051051 4y ago
> The ideas behind the DevOps movements undeniably changed the software development world for the better I absolutely deny this.
mikkergp 4y ago
This seems like an example of something similar to the no true scotsman fallacy. (it's inverse) These ideas are supposed to be aspirational, akin to asking "is it a true democracy" or "is it really capitalism". I don't entirely object to the notion of trying to be more precise, but gatekeeping "devops" or "agile" also seems to sort of miss the point.
rr888 4y ago
For me devops was never a great idea. Ideally you want one team to decide on the architecture. You do need front line ops, ideally they'd take some ownership and write some scripts, but the buck stops with one team for inhouse systems that is usually developers.
manuelabeledo 4y ago
It kind of feels like devs have been duped for the past twenty or so years. Want to save in project management? Just make devs responsible for planning and estimating, and call it "agile". Want to remove sysadmins and ops? Make devs write their own configuration for the infrastructure, and say you do "devops". Want to eliminate QA? Say that you do TDD.
- 0x457 4y ago
  > Want to eliminate QA? Say that you do TDD. I don't think anyone ever thought of it like this. Well, some dumb managers probably did. If you want to eliminate QA people, then you need an enormous automated E2E test suite. To have that, you need dedicated people who think of and write those test...one would call it a QA team.
louwrentius 4y ago
This is a content-free extremely meager blog post that doesn’t really belong on the front page of HN but the discussion in the comments is quite illuminating. The discussion shows how a lot of people have wildly different ideas about what devops is and that in and of itself is interesting.
torginus 4y ago
I'm confused about DevOps - I thought it was about dev teams owning the infrastructure, testing and deployment pipelines, so that there is no clear delineation between they person who writes tests, the person who sets up CI/CD pipelines and the person who codes the app. It has nothing to do with frequency of deployments, and it does not mean that no ops team is necessary for running the show, it's about team autonomy and better understanding of different teams' concerns because everyone is doing a bit of everything.
RHSeeger 4y ago
I find the article somewhat ridiculous in that it takes the tone that everyone thinks the idea in question are good. > They aren’t convinced that some of the scarier ideas (like deploying multiple times per day) will help after all For some cases, rapid deployment might make sense (purely front end stuff). For a lot of cases, I'm firmly of the opinion that it does _not_ make sense. I've never worked on a project that deployed that often, and I don't feel the need to. I have depended on services that deployed that often and "oh, sorry we broke that, we'll fix it later; you'll just have to live with permanently broken data" gets my knickers in a twist every time. > I don’t buy into all the ideas behind agile, particularly some of the Scrum-specific ceremonies. That's because a lot of the Scrum people present it as "the" Agile. And a lot of the non-Scrum people are fairly convince that Scrum is awful. To which the Scrum people present "no true Scotsman" arguments. I'm all for the underlying "listen to your customer, listen often, and change plans as often as necessary to make them happy" ideas of Agile. But every "methodology and process" for it I've seen completely loses that idea. It's infuriating.
- 0x457 4y ago
  I think your issue isn't rapid release cycle, but lack of proper testing.
  - RHSeeger 4y ago
    Yes, but testing takes time. Automated testing is a wonderful thing, but it only catches those things that the testers have already though of AND have had the time to write automated tests for. Some things are notoriously difficult to automate testings of. And there are always things that don't get considered during the writing of tests; especially those things that only appear when multiple systems are interacting together. Taking the time to deploy code to a staging server, where QA can check it, and other systems can interact with it, can prove to be very beneficial. And runs contrary to having every push (or every push that passes the automated testing suite) automatically go to production. I'm a big fan of something like a 2 week release cycle, plus the ability to release quickly when needed (generally same day, but only when necessary). I'm a big fan of automated tests (both unit and integration, both dev and QA written), plus a QA person/team to bang on things in a way developers don't think of, plus a staging environment where all the (staging versions of the) system interact with each other. I'm not a fan of pushing things to production willy nilly, with a high risk of breaking things "because you can push again with a fix in an hour".
GuB-42 4y ago
The problem with these fad methodologies is that it either works, or you are not doing it properly. The methodology can't be wrong. It is like saying that C programs never crash, because if they crash, it means you are not using C properly. It is true, but no one will pretend that, everyone knows that under real life circumstances, C programs can crash, and there are plenty of other programming languages that address this issue, at the expense of other things, like performance or simplicity. But for some reason those who push methodologies don't seem to consider that if that methodology is difficult to implement, or if it is not robust against things like conflicts of interest and office politics, or whatever, then maybe it is a problem in their methodology. It doesn't mean the methodology is bad, just that it doesn't work in every case, and that some tweaking may be necessary.
PedroBatista 4y ago
> I think companies feel that if they aren’t going to “do DevOps”, that they need to rebrand the term “DevOps” to mean “what we happen to already do”. > To me, the most egregious example of this is when a company simply changes the job titles of all its ops people. John use to be an “IT Operations” guy, abut now he’s a “DevOps Engineer”. This is so true it hurts..
mekoka 4y ago
> We need better definitions for these terms When I point to the moon with my finger and say "moon", the fool looks at my finger and calls it the moon. Is it more important to mindlessly tag yourself "Agile", with all the bells and whistles adorning the capital A, or would you rather just be agile according to the sought out goals and philosophy, and adapted to your specific situation? Is it more important to have a clearer definition of "DevOps", so that you can more accurately implement all the prescribed workflows, or would you rather understand the pursued ideals, so that you can pick and choose what you need, according to your specific context, unencumbered by a slew of best, but ultimately useless, practices?
freshnode 4y ago
I think there is a third category - companies that have heard of DevOps, think it means that the development team and operations team talk to one another, which they already do (!) and shut down conversations after that. I've come across this archetype in several contracting gigs and I find it to be especially pernicious as it gives the incumbents (who usually have more weight in the org) ammo to effectively do nothing.
NonNefarious 4y ago
eweise 4y ago
I only do as much 'ops' work as required by the idiots in charge. Its much more productive for me to be working on business solutions than screw around with getting traces and logs into datadog and figuring out all the permission issues in AWS. Someone can specialize in that knowledge instead of everybody needing to know everything.
quantum_state 4y ago
Just being devil’s advocate, whatever the profession is, if someone keeps changing things, how would it reflect on his or her maturity in getting things done?
EricLeer 4y ago
I wonder how to use a devops flow for machine learning workloads. Something we are working on at my current job is how to balance the speed of releasing together with keeping data, pipeline and model versions aligned (where it for instance takes multiple days to release a new model). Releasing new features which need a model retraining is currently a very jarring experience.
ruh-roh 4y ago
While I generally agree about the annoyance with UI changes, and how that grinds on people, I hope we don't just handwave over the "CD for bug fixes is fine" bits. It varies product-to-product obviously, but I'd guess there is also a significant chunk of folks that are pleased that their functional annoyances/blockers (bugs) can be fixed quickly, without waiting for a monthly or quarterly release. These folks might not mind the fluctuating UIs, if it means they get the functional value they need asap.
not_me_ever 4y ago

235 points by wagslane 4y ago

If you have a separate DevOps role in your org, arguably you're not doing DevOps. So probably most orgs aren't doing DevOps, at least those that are talking about it.

draw_down 4y ago

I see the point, but it’s funny that the author lays out the reason why this goalpost-moving is happening. As pointed out, you sound like an idiot if you’re against Agile. So it’s a lot easier to get what you want if you just call it Agile. I get that it’s not intellectually rigorous or whatever, but good grief, when has that ever mattered in business. Is there any class of concepts that survives unblemished after contact with reality?

tuckerman 4y ago
I'm not sure I agree with the author wrt Agile. I've never been on a team that did anything that resembled agile but I have been on teams that actively pushed back when anything that looked like it sort of resembled agile got floated. I'm not saying its 100% bad but I'm not sure it has a universally good reputation
- draw_down 4y ago
rufflez 4y ago
Agile is bogus nonsense - there I said it.
- latchkey 4y ago
  Cool... what's your alternative?
  - rufflez 4y ago
    Software used to be built before Agile... Let's at least keep solution design and also get senior engineers to help estimate And do away with daily stand up and bs rituals.. a meeting is needed when it is needed, not for the sake of "following the process"
- kenrose 4y ago
  Yeah, iterate quickly, learn from your customers usage, repeat. You're right, total rubbish. /s
  - stonogo 4y ago
    Your pithy dismissal leaves out hundreds of pages of religious rites and project management dances which invariably turn into productivity-sucking meeting overload, unachievable goals, and constant product churn, effectively alienating everyone from the developer to the user, leaving only the managers satisfied. Like telling a college student that communism doesn't work, any objection is met with "well they're just not doing REAL Agile," skipping over the fact that nobody on Earth seems to be "doing REAL Agile." Agile could just as well serve in the headline in place of DevOps.
  - rufflez 4y ago
    This is not how it works in real life. Contracts are made to the lowest bidder, there are promises to keep or heads will roll. By iteration 10, your customer is fuming that the half baked idea has more exceptions than handled scenarios and the finger pointing begins. Also, forget about architecture, performance, security and other pesky little problems. I have never seen it play out differently. The biggest problem is that the person footing the bill does not care about your story points or retrospective- and I don't blame them for that
drewcoo 4y ago
> Is there any class of concepts that survives unblemished after contact with reality? We're not talking about reality, though. We're talking about things consultants market to management.

itsmemattchung 4y ago

I was one of the biggest proponents of DevOps (even started and the DevOps orange county meetup) but ... after working at AWS for 5+ years, my entire perspective shifted. While DevOps does work in some organizations, sometimes what you really need to do is operationalize your software development team. Don't get me wrong, system developers/engineers/operations are needed but for sometimes for a small four person start up, the best investment is improving the development team's operational posture (e.g. run books, alarming, monitoring).

0x457 4y ago
> for a small four person start up, the best investment is improving the development team's operational posture (e.g. run books, alarming, monitoring). But that's literally what DevOps is? DevOps isn't a special DevOps team that writes terraform.
calvinmorrison 4y ago
> development team's operational posture (e.g. run books, alarming, monitoring. Isn't DevOps in large part forcing the developers to have responsibility over the code they push and giving short and frequent pushes to easily rectify problems? Making developers do support work reduces support in the long run because they'll get frustrated and automate solutions.
- stetrain 4y ago
  Yeah, I feel like the term has been taken over to mean using kubernetes and having continuous deployment pipelines for your microservices. My understanding of root of the term is about ownership and responsibility, where the team that develops the app also owns how the app deploys and runs in production, including monitoring and responding to issues. If the team decides that the right way to deploy their app is once every two weeks using a shell script and a zip file, cool. As long as it's documented and monitored so you know if it's working.
  - calvinmorrison 4y ago

ciguy 4y ago

DevOps has lost all meaning as the first paragraph in the article says. Anything it says after that is just semantics. Everyone is doing it on some level, but what that means to each company differs widely.

mnd999 4y ago

All these processes are a template, a starting point. Begin there and optimise and refine to suit your team and your environment. Dogmatically following a process that doesn’t work for you is silly.

paxys 4y ago

I worked at a large internet company in the mid-late 00s, and the typical team composition was a handful of developers, a product manager, a designer (if you worked on customer facing UI), a project manager (for large enough projects), QAs, and a SRE. Every team had dedicated engineers whose sole purpose was to place purchase orders for servers, set up and manage databases, deploy changes, monitor uptime and more. And this was standard practice across the industry, as soon as your company outgrew a shared host. Even the thought of something like this would be absurd today. So I'd flip the "no one admits they don't to it" around – your company does DevOps already, whether you realize/admit it or not.

dilyevsky 4y ago
Replace “purchase orders for seevers” with “apply terraform to spin up VMs” and ditch the QA person and not much has changed
- deleted 4y ago
  [deleted]
water-your-self 4y ago
In the mid 2000s was the role referred to SRE yet? Facebook calls it production engineer, DevOps is broad/ im unsure its origin, Google calls it SRE.
- jbverschoor 4y ago
  Within non-tech companies, they call everything "the programmer"
jzawodn 4y ago
Yahoo?

ipnon 4y ago

SRE seems the better approach. Make an error budget, measure reliability, stop releasing in favor of reliability engineering if the budget is missed, use the free time reduce toil. DevOps seems to degrade to disorganization because the incentives are not aligned between product development and software engineering.

jameshart 4y ago
“Make an error budget, measure reliability, stop releasing in favor of reliability engineering if the budget is missed” These are all things that fall in the realm of ‘ops’. ‘Dev’ops is an approach to ‘doing ops’. It is one that adopts certain practices that are common in ‘dev’elopment - such as defining things programmatically, using automated repeatable processes to generate work product, and using automated testing and verification. So what you describe as ‘SRE’ doesn’t seem to me to be ‘not devops’. Seems like a set of ideas that would benefit from a devops kind of approach.
tootie 4y ago
I think you've hit at the heart of the problem, but swapping the DevOps model for SRE is robbing Peter to pay Paul. Last place I worked with a 100+ person tech team had more of an SRE model. The SRE team were charged with reliability. They had no incentive to delivery features or to improve developer experience. As a result, developer tools sucked, there was s rigid release schedule, every bug was met with a rollback. This was probably an extreme case because the team was also not very good, but either way it was a clear case of misaligned incentives. The entire point of DevOps was collective ownership with a shared goal of customer value that is function of feature richness, quality and reliability. It's the Holy Grail of efficiency, it's just really hard to build that kind of culture.
- ipnon 4y ago
  Seems like the sweet spot is DevOps style reliability ownership with SRE style indicators, objectives and agreements.
jedberg 4y ago
If you think SRE and DevOps are opposed to each other, then you might be doing one of them wrong. I headed up SRE at Netflix, and our sister teams wrote the software and managed the tools that enabled the developers to continuously deploy and operate their own services. We were in charge of reliability, but we did it through chaos testing and accountability, and enhancing the deployment tools to warn devs that they might be doing something high risk, but still letting them do it.

drewcoo 4y ago

The entire article avoided talking about definitions of DevOps, while rambling a lot about how we have different ideas of what it means and how we need better definitions. I appreciate the alert that there's a problem here, but there's no substance to the article.

vesinisa 4y ago

The author is right that DevOps seems to often mean different things to different people. I would have loved it if they had taken more of an attempt at one definition than just that it's when developers have an understanding of how and where their code is deployed. That does sound sensible, but what other best practices does DevOps usually entail?

giantg2 4y ago

I feel like DevOps is a trap. Most companies that have sufficient talent and organization/operational setup don't see a need for DevOps. There seem to be a lot of other companies that are sucked in by its promises and think that it will solve their issues. Surprise... this didn't fix your shitty policies, disorganized/unempowered org structure, or cheap talent (cause you're unwilling to pay or train). I assume there are a few places that can benefit from this. Maybe some medium to large tech companies with great procedures, documentation, and an empowered org structure. Especially if they're paying enough for people who can manage it. The other is probably for very small places wirh fairly simple tech where you can't afford entire teams of people so you empower someone to really own the system top to bottom (without ever calling it DevOps).

turtlebits 4y ago
IME, letting developers have free reign over service architecture/deployment/CI/observability is a bad idea. Especially when every team comes up with their own option. You need some sort of Devops (best practices if not a team) to provide guidance and help implement standards for these.
- giantg2 4y ago
  I agree, if we're talking about places big enough to have multiple teams. The thing I was pointing out is that most places that have multiple teams are terrible at setting up best practices. They'd be better off with a centralized "environments" team provisioning the server, network, etc so the dev teams can work on the actually implementation. At least it's easier to maintain standards, or at least consistency, across apps if the low/mid level config is handled by one team. When I was talking about small places, I mean like tiny mom and pop non-tech companies with an IT department of 1-5 people.
ryukoposting 4y ago
Every company that has a software product also has DevOps, whether or not they call it that. If someone manages your cloud infrastructure, they're doing devops. If someone is coming up with CI/CD setups, they're doing devops. If someone came up with SOP for distributing new releases, they're doing devops. Whether or not anyone's title has "DevOps" in it, devops tasks are still devops tasks. By the same logic, even though I watered the office plants this morning, I am still a firmware engineer- not PlantOps. The real trap of DevOps is the assumption that it's anything new or non-essential. DevOps has existed since the notion of "software as a product" came into existence, we just didn't have a name for it yet. None of this is to say you need a dedicated DevOps person (or PlantOps, for that matter). DevOps is an integral part of any software product, even if no specific member of the development team is dedicated solely to DevOps tasks.

ebiester 4y ago

Or, alternatively, it might not be a great idea in all cases. Maybe your development team shouldn't develop terraform the four times a year they need it. It might even be faster to have two or three individuals with a singular vision for the system in a smaller team. (There may not be enough work for some teams to have someone with specific DevOps skills. However, that ends up in a partial-shared-service model where team members get called off for organizational concerns.) Maybe you need a subject matter expert that will develop standards for teams and select a set of tools, setting those up so that teams can build off those for their needs. (This is why Ops are often shared services teams.) Maybe it's not a great idea if the first team to encounter a problem picks the tool that works best for them, and the organization ends up with multiple tools because there was nobody with both the time and expertise dedicated to the analysis. (This is why Ops are often shared services teams.) We need to break down barriers. We need to adopt valuable practices. However, the parts of DevOps that don't make it to the real world may just be because they don't solve the problems of the organization.

JohnFen 4y ago

As a dev, the whole "rapid release" thing made a lot of sense to me. As a user, though, I've turned 180 degrees on it. I hate "rapid release" with a passion now. Which means I dislike Agile by consequence. Rapid release means I can no longer rely on the software that engages in it. It's constantly shifting, with frequent changes in UI and workflow, and new and exciting bugs constantly appearing. I miss the days when I could plan ahead for new versions, so I could time them to have the least possible disruption to my life.

cjsawyer 4y ago
I miss being able to actually refuse an update. Or even being able to disable them!
- dleslie 4y ago
  I miss knowing that updates even happened. I also miss important features remaining stable. Which is why, for the most part, I've stopped using web-based applications.
  - ethbr0 4y ago
    You want a chuckle, turn off automatic app updates on an Android phone and see how many prompts you get. And how terrible the patch notes are ("Fixed some things")
    - dleslie 4y ago
      In fact, my Android 11 device has no browser enabled, the play store is disabled, and almost every app it came with is disabled. It functions solely for email, instant messaging, a TODO tracker, and taking photos. Edit: not sure why this was downvoted. Did someone find the idea of a narrowly-focused phone to be offensive or unworthy of discussion?
      - xwolfi 4y ago
        We know but OP was telling you to experiment: turn off auto update and turn on the play store, and see how many prompts you get. We're not sure why you went to describe how you read emails, and that's why we downvote you. Your answer was completely irrelevant !

the_jeremy 4y ago

My team at Nasdaq (I work on other products, not the stock exchange) does DevOps surprisingly well for not being a Real Tech Company TM. I don't know that the business sees benefits, as it meant that all of the devs on my team have had to learn Terraform, Helm, AWS, and Kubernetes in addition to the actual SpringBoot Java application code, so our feature velocity dropped quite a bit, but it was fun to learn, and it's probably better to have people who understand the full deployment and not just pass blame between teams during incidents.

lmarcos 4y ago
Oh man, that's the trap. I do like to learn new things, I swear. But at work I want to focus on what I do best: feature development. Sure, I do know terraform and k8s and what not (because I play with them via my side projects), but at work don't give me the Ops/Platform hat please! It's cheap labor (your company doesn't want to hire a dedicated platform engineer to handle terraform/k8s/etc. and instead assigns you (the developer) that responsibility claiming "you are gonna have fun learning" and paying you 25% more in the best scenario)
- MetaWhirledPeas 4y ago
  I don't know if you're right or not, but I do think your hesitancy mirrors why a lot of organizations are unable to adopt DevOps fully: many people don't want to wear a bunch of hats. They just want to do their one thing, and I can't really blame them.
  - tremon 4y ago
    But I don't think the original DevOps (TM) specified that every person was required to work on every part of the stack? Just that you need to have full-stack skills and responsibilities within the same team? That said, I probably wouldn't want to hire someone with just one hat. It's ok to have a preferential hat, it's not ok to refuse to wear any other.
    - BlargMcLarg 4y ago

mkl95 4y ago

Most SaaS are overcooked spaghetti floating on AWS alphabet soup. A devops mentality won't fix them.

robertlagrant 4y ago

> If I were to generalize, I’d say most of the companies I’m picking on fall into one of two categories: > They believe in DevOps ideas, but they’re unwilling to invest in changing their ways. They aren’t convinced that some of the scarier ideas (like deploying multiple times per day) will help after all. Option number 3 is the biggest one for me: thinking that having a separate "DevOps" team from the Dev team that does ops via modern tools is DevOps. That may be a good approach for your company, but it ain't DevOps.

spiralpolitik 4y ago

DevOps (and Agile) both usually run aground on the same question of trust. If you aren't willing to trust your delivery teams then you aren't going to see the leverage that both practices bring to an organization. And if you aren't willing to trust your delivery teams then your organization has a much bigger problem than either DevOps or Agile can solve.

tibbon 4y ago

I also go to the gym 4 times a week, go to sleep on a regular schedule and eat a balanced diet!

alexjplant 4y ago

According to "The DevOps Handbook" DevOps is... > the outcome of applying the most trusted principles from the domain of physical manufacturing and leadership to the IT value stream. DevOps relies on bodies of knowledge from Lean, Theory of Constraints, the Toyota Production System, resilience engineering, learning organizations, safety culture, human factors, and many others. Anybody that's worked in an organization that "uses DevOps" or has "DevOps Engineers" knows that this simply isn't true in the vast majority of circumstances. Too often DevOps is just a new name for an ops team. "True" DevOps is all about aligning incentives, sharing responsibilities, and fostering collaboration between people who build and maintain systems whereas "DevOps" (with air quotes) is just a new spin on being a sysadmin with a greater emphasis on AWS and Terraform.

baobabKoodaa 4y ago
Both the quote and your response to it read like word soup to me. I have literally no idea what the point in either of these are. Historically, the word "DevOps" meant that instead of having separate "Dev" and "Ops" teams, the same people would do both Dev and Ops work.
- lazide 4y ago
  Which is what they’re saying. Because they do both jobs, folks understand both sides, and work together to make things work better end to end for them and for customers. Alternatives include siloed dev teams that throw shitty software that crashes in production all the time to an overworked ops team that takes all the blame. And overworked ops teams that have gone pager-blind and only care if a VP yells at them, so ignore all but the most major issues - after they already happened.
  - 0xbadcafebee 4y ago
    The fact that engineers are not taught those topics are why DevOps has failed. We're expecting whole organizations to magically improve when they don't understand how. I can count on one hand the number of people I have met who understand OP's comment.

rhacker 4y ago

No one admits they don't do it because everyone does it. I have worked at 6 different companies since I've heard that term and all of them were able to deploy to an environment by clicking a button on a CI/CD tool. Maybe I'm the outlier?

0xbadcafebee 4y ago
Anyone can deploy a file with one button. DevOps is what happens when the app starts randomly crashing. If the site keeps humming along like normal, or the app is automatically rolled back, or (rarely) the devs get a page & quickly diagnose in production and apply a fix in 10 minutes, and deploys don't randomly fail due to unpinned versions etc, that's DevOps. If people start running around like chickens with heads cut off and pointing fingers, that's not DevOps.
- leetrout 4y ago
  I like this distinction. I think it succinctly captures a big difference.

vinaypai 4y ago

Not directly related to the question, but I personally hate the term "Sprint". Physiologically, a sprint involve by definition is a short-term high effort activity that can't be sustained without a break. Does anyone use a different term that doesn't imply working oneself to death?

duped 4y ago
Cycle
0xbadcafebee 4y ago
Just changing the word isn't going to change the practice. You're supposed to take breaks between sprints. There needs to be planning and alignment to an easily-obtained goal, which requires time. But most people don't know how to do Scrum and hand-waive the goal, the planning, the retro. The end result is unsustainable work with no specific goal in mind that exceeds the sprint cycle. Or 20 epics all in progress at once. Or business goals that shift and put unobtainable expectations on teams.
- bot41 4y ago
  What's a break period? Where is that described?
  - 0xbadcafebee 4y ago
    It's not explicitly described. "The break" (imo) is giving enough time to planning at the beginning, and actually dealing with the review & retro items. The length of the sprint can be shortened so that there is more learning and planning. Sprints are cancelled if the goal becomes obsolete.
0x457 4y ago
Cycle or stint. I like "stint" more.

wiremine 4y ago

Recently I've been reading "When Agile Gets Physical" [1] and "Extreme Programming Explained": 1. "Extreme Programming Explained" explains the importance of separating values from principles and practices. 2. "When Agile Gets Physical" unpacks why using agile software best practices are horrible for hardware. This is "duh" if you're a hardware person, but the book goes back to agile/lean first principles, and then builds up an approach for agile hardware. It got me thinking that we often conflate values and practices. Blinding adopting Scrum or DevOps without understanding some of the underlying value and principles leads to problems. If you understand the foundational principles, you're in a much better position to adopt/reject certain practices for your given situation. In reality, most organizations just adopt a few agile practices, while rejecting the basic values. [1] https://rapidlearningcycles.com/when-agile-gets-physical/ https://rapidlearningcycles.com/when-agile-gets-physical/ [2] https://www.amazon.com/Extreme-Programming-Explained-Embrace-Change/dp/0321278658 https://www.amazon.com/Extreme-Programming-Explained-Embrace...

dec0dedab0de 4y ago

I was a network engineer at my previous job, I was fed up with doing things manually so I learned to code better. Now I'm a full stack developer writing code for the network team of a different company. One thing that has changed drastically over the last 15 years is the sharp divide between IT and development. It used to be that developers were clueless how to deploy their own apps, and IT was terrified of writing code preferring to do things manually, or purchase something prepackaged. Now I see developers that understand firewalls and load balancers, and IT folk that can script stuff out for themselves. I think we have the push behind DevOps to thank for that progress, and while there is still a long ways to go, I think it's a good thing overall.

mxuribe 4y ago
> ...writing code for the network team of a different company... Something like is the kind of job that i think i want to do next. It need not be for a network team...but i'm headed more for back-end work and these days i rather engage with internal teams. Separately, to your comment on sharp divide...The sharp divide that i have seen is a lack of cohesion for the following 4 IT groups: first-level support which gets sent to off-shore; the folks who (honestly have the most face time with internal clients who) roam around the office fixing desktops and building network/wiring, and setting up/fixing A/V equipment in conference rooms; sys admins; and developers. Furthermore, in the last 2 companies that i worked for, there has been a push to "centralize" certain functions like IT and HR to "shared services"...but at the end, none of these groups seem centralized! They all now work more separated than ever before...and most of the interaction between each of these groups - and even between the 4 subgroups i noted above in IT alone - mostly communicate via tickets, or some other thick-layered mechanism. Sure, they've been centralized in that these people were pulled from some respective business unit, and brought under a more single umbrella...but in the end, these workers hate it, the effecicny drops, more bureacracy is created (which creates no value)...and some executive gets to lead the shared services...and of course some management consultant company gets their payday. Oh, and yeah, IT at these orgs definitely does not use any of the good parts of devops, agilescrum whatever. Some orgs try and adopt some cookiecutter approach only to have it create yet more bureacracy...but "hey, we're agile!" /sigh

jbverschoor 4y ago

Devops means let a non-admin developer have access to aws and just buy everything he can find in order to try and fix his shortcomings. It'd be better and cheaper to follow a sysop course

nkotov 4y ago

The mistake was turning DevOps into a job position which resulted in silos being created more when the whole purpose was to remove/prevent silos and to allow developers to operate at their own pace.

honkycat 4y ago

"DevOps" is a corrupted term. It just means sysadmin at this point.

mikkergp 4y ago
While I get the underlying argument you are trying to make, there are definitely still sysadmins in the world who see their job as administering systems, and refuse to look at anything resembling automation.
bot41 4y ago
I think it means devs that write their pipelines themselves. Someone else has to enable the devs to do that

synu 4y ago

I feel like devops became a marketing juggernaut then somehow evolved into creating a third team, apart from dev and ops, that does handoffs with both of them.

jascii 4y ago

I think one of the problems with the terms "Agile" and "DevOps" is that they are pretty general and generic terms that they are trying to cast very specific meaning too... “So you’re saying you’re not an agile team?” Sure we are appropriately agile for our needs, no need to be rigid about it...

notabee 4y ago

Corporate kayfabe. https://en.wikipedia.org/wiki/Kayfabe https://en.wikipedia.org/wiki/Kayfabe

nont 4y ago

This is also very true in the presales engagements by top consulting firms under "Digital Transformation". These presales teams would come and show, on paper, how great they are with the agile practices, devops, microservices, you name it. However, it's a totally opposite story during the actual implementations. I've witnessed that it took a top consulting firm 4 months to provision a sandbox environment while being 1 month away from the first release. When asked about their proud DevOps practices, you'd hear excuses that are in the blatant lie territory. They knew they could get by by managing the stakeholders instead of really doing what they had promised.

0xbadcafebee 4y ago

I agree the terms have become pointless. I have been thinking about opening an actual school for DevOps, or at least IT Ops. There is virtually no education for it, but it's a complex subject matter that requires years of experience. The SRE books are a peek under the covers but leave so much unsaid that it's really only useful at "doing operations", not digital transformation. To properly build and run a high-performing organization, you need to understand at least 1) the entire history (up to today) of software development practice and production operations, 2) all of the competing requirements of product design, development, and operation, 3) architectural best practices, 4) the industry standards that apply to the work, and 5) tech product-aligned business management methods. Basically you need a generalist who has 15+ years experience and has tried and failed at everything under the sun. I think of it like carpentry before there was a building code, and before the introduction of apprentices and licenses. Trying to build high-performance orgs without a master builder to show you is like trying to build skyscrapers after having built your own home (and having never seen a skyscraper). Or worse, taking an old barn (non-tech enterprise) and trying to build a skyscraper out of that. Without a trade school it will continue to be just a craft.

benreesman 4y ago

When I was a wee lad, “programmers” wrote software and sorta tested it and then kicked it over the fence to “systems administrators”, who chain-smoked and were very, very un-woke. This was a bad system. To the extent that “DevOps” means anything, it’s that the spheres of capability and responsibility need to have tons of overlap: SWE’s need to be in the oncall loop and know how the filesystem works, and SRE’s need to be serious coders. That’s it: everyone codes, everyone operates. We all automate where feasible. Sidebar: What’s with the crazy surge in BuzzFeed-style titles on the front page? It’s not a federal matter but it’s not a slight change YoY either.

kazinator 4y ago
The test cases for Unix must have been kept on some tapes that were lost, because a test case is hardly to be seen in the historic sources you can find online.

oxfordmale 4y ago

Agile is a philosophy, it is not sprints, story points or whatever crazy methodology an expensive Agile Coaching company has come up with. The same applies to DevOps. DevOps could just be a bash script, written by the developer, to feit their codebase. It doesn't have to be fancy or require an extensive amount of investment.

mxuribe 4y ago
You sound like Mr. Miyagi from the Karate Kid movies (which is a good thing!)...specifically it remains me of that scene when Daniel-San asks him what "belt" he is...and Mr. Miyagi responds "Calvin Klein"! (At some point, Mr. Miyagi explains that his exppertise of Karate is only for the need to survive encounters, and not worry about which "belt" he had achieved.)

irrational 4y ago

I hate devops. Mainly because I don’t want to do it. I liked it better when there was a dedicated infrastructure team instead of pushing it down to developers and giving it a new name.

hedora 4y ago
Yeah. As a developer, I need a competent QA and operations team to support me. I'm not competent at those things, but 90% of developers I've worked with are somehow worse at them. I also need 1-2 months between major context switches, and the ability to declare a subcomponent "ready for release". Moving forward, I will actively avoid companies that claim to use agile, scrum or devops. Edit: Also, I want to work on a team that has a good release manager.

MetaWhirledPeas 4y ago

> We need better definitions for these terms How about this? Let's describe levels of adoption. Think of the "normal form" concept for databases. Each level has a strict definition, and you can objectively observe which level you are using. And not every level is appropriate at all times. So maybe a team could say, "we're at DevOps level 2," or, "we use level 3 Scrum."

leetrout 4y ago
Yes. I like this concept a lot. Maybe accompanied with a landscape of verticals and how you progress in each one.

AtNightWeCode 4y ago

You go up at 5 AM in the morning. You ride your bike to a customer. You pass some security checks like, known guy, yes, no. You are shown to the servers in the cellar. Cause they are safe there. You get access to the admin account after 2 hours. You do some backup to a local disk. Somebody screams something but you can’t hear what because of the noise. You say yes and hopes it was about coffee even though the room is like a sauna. Now you pick up that CD and insert it. You are not allowed to use a USB-drive because it is mutable. Some clicks and the installation begin. And now you wait. I am pro-devops.

nimbius 4y ago

when John Willis explained it in 2016 devops had enormous potential for the initiated. id guess maybe 10% of anyone who says 'devops' does anything remotely close though. devops in 2022 means if youre a dev, youre expected to know the bowels of the OS inside and out as well and be willing to spend hours debugging it alongside your normal programming work. if youre in ops? it means debugging someones undocumented middleware from nine years ago while the front of site burns so "rockstar" coders dont jump ship having to do it instead. if youre in dev or ops, it now means standups where you sit down for an hour and talk about where to put your story in jira or how to enter it at all, and where you discuss broad strategy too. instead of leaving after your status update, youre expected to stay. its just another meeting now. and once boardmembers and leadership realized they could spare themselves the crucifixion of any real accountability after a breech, they turned devops into devSECops, meaning your infosec team sits in awkward silence as you explain why double defined nginx headers arent a feature and the same management team that made them part of the standups also unilaterally declared ssl3 can never be deprecated because customers.

encryptluks2 4y ago
This is my issue. Companies have commoditized the average developer and now treats it like a average entry level role

jammycakes 4y ago

In a previous job, I joined a team that was supposed to be introducing DevOps to the organisation. It started out well -- we spent a few months hacking with Terraform, Docker, Vagrant, Kubernetes, and related technologies to implement an infrastructure-as-code approach -- automating the process of provisioning and decommissioning servers, and building a setup where development teams could deploy updates with a simple git push. Unfortunately it all went downhill fairly rapidly. We ended up spending the majority of our time manually applying security patches to a bunch of snowflake servers that we'd lifted-and-shifted from another hosting provider to AWS, and fielding support requests from the development teams. Within a year, we were being told in no uncertain terms by our project manager that we were an operations team, not a development team. It felt like a complete bait-and-switch. Within two years, I had left the organisation in question and moved on to a new job elsewhere doing actual development again. Last I heard, the entire team had been disbanded. It sounds like the author of this article must have had a very similar experience. I wonder just how common it is. It seems that in many places, "DevOps" is all Ops and no Dev.

spaceywilly 4y ago
> We ended up spending the majority of our time … fielding support requests from the development teams This has been my experience at 3 different small-medium companies now. A too small DevOps team suddenly is in the critical path for even the most trivial software task, then engineering productivity grinds to a halt. I think a much better pattern would be to enable dev teams to self serve. Set up the required infrastructure and guard rails, then let teams handle their own deployments and infrastructure. Give people what they need to do it themselves instead of having to open a support ticket for everything.
- lazide 4y ago
  Most developers won’t know how to do it without footgunning themselves constantly is the problem. If the dev ops team is staffed enough to develop integrations that won’t allow that AND won’t get in the way, and then train folks and ‘keep the line’ enough to stop the scope creep - they’re probably not at a shitty small/medium sized shop.

g051051 4y ago

> The ideas behind the DevOps movements undeniably changed the software development world for the better I absolutely deny this.

mikkergp 4y ago

This seems like an example of something similar to the no true scotsman fallacy. (it's inverse) These ideas are supposed to be aspirational, akin to asking "is it a true democracy" or "is it really capitalism". I don't entirely object to the notion of trying to be more precise, but gatekeeping "devops" or "agile" also seems to sort of miss the point.

rr888 4y ago

For me devops was never a great idea. Ideally you want one team to decide on the architecture. You do need front line ops, ideally they'd take some ownership and write some scripts, but the buck stops with one team for inhouse systems that is usually developers.

manuelabeledo 4y ago

It kind of feels like devs have been duped for the past twenty or so years. Want to save in project management? Just make devs responsible for planning and estimating, and call it "agile". Want to remove sysadmins and ops? Make devs write their own configuration for the infrastructure, and say you do "devops". Want to eliminate QA? Say that you do TDD.

0x457 4y ago
> Want to eliminate QA? Say that you do TDD. I don't think anyone ever thought of it like this. Well, some dumb managers probably did. If you want to eliminate QA people, then you need an enormous automated E2E test suite. To have that, you need dedicated people who think of and write those test...one would call it a QA team.

louwrentius 4y ago

This is a content-free extremely meager blog post that doesn’t really belong on the front page of HN but the discussion in the comments is quite illuminating. The discussion shows how a lot of people have wildly different ideas about what devops is and that in and of itself is interesting.

torginus 4y ago

I'm confused about DevOps - I thought it was about dev teams owning the infrastructure, testing and deployment pipelines, so that there is no clear delineation between they person who writes tests, the person who sets up CI/CD pipelines and the person who codes the app. It has nothing to do with frequency of deployments, and it does not mean that no ops team is necessary for running the show, it's about team autonomy and better understanding of different teams' concerns because everyone is doing a bit of everything.

RHSeeger 4y ago

I find the article somewhat ridiculous in that it takes the tone that everyone thinks the idea in question are good. > They aren’t convinced that some of the scarier ideas (like deploying multiple times per day) will help after all For some cases, rapid deployment might make sense (purely front end stuff). For a lot of cases, I'm firmly of the opinion that it does _not_ make sense. I've never worked on a project that deployed that often, and I don't feel the need to. I have depended on services that deployed that often and "oh, sorry we broke that, we'll fix it later; you'll just have to live with permanently broken data" gets my knickers in a twist every time. > I don’t buy into all the ideas behind agile, particularly some of the Scrum-specific ceremonies. That's because a lot of the Scrum people present it as "the" Agile. And a lot of the non-Scrum people are fairly convince that Scrum is awful. To which the Scrum people present "no true Scotsman" arguments. I'm all for the underlying "listen to your customer, listen often, and change plans as often as necessary to make them happy" ideas of Agile. But every "methodology and process" for it I've seen completely loses that idea. It's infuriating.

0x457 4y ago
I think your issue isn't rapid release cycle, but lack of proper testing.
- RHSeeger 4y ago
  Yes, but testing takes time. Automated testing is a wonderful thing, but it only catches those things that the testers have already though of AND have had the time to write automated tests for. Some things are notoriously difficult to automate testings of. And there are always things that don't get considered during the writing of tests; especially those things that only appear when multiple systems are interacting together. Taking the time to deploy code to a staging server, where QA can check it, and other systems can interact with it, can prove to be very beneficial. And runs contrary to having every push (or every push that passes the automated testing suite) automatically go to production. I'm a big fan of something like a 2 week release cycle, plus the ability to release quickly when needed (generally same day, but only when necessary). I'm a big fan of automated tests (both unit and integration, both dev and QA written), plus a QA person/team to bang on things in a way developers don't think of, plus a staging environment where all the (staging versions of the) system interact with each other. I'm not a fan of pushing things to production willy nilly, with a high risk of breaking things "because you can push again with a fix in an hour".

GuB-42 4y ago

The problem with these fad methodologies is that it either works, or you are not doing it properly. The methodology can't be wrong. It is like saying that C programs never crash, because if they crash, it means you are not using C properly. It is true, but no one will pretend that, everyone knows that under real life circumstances, C programs can crash, and there are plenty of other programming languages that address this issue, at the expense of other things, like performance or simplicity. But for some reason those who push methodologies don't seem to consider that if that methodology is difficult to implement, or if it is not robust against things like conflicts of interest and office politics, or whatever, then maybe it is a problem in their methodology. It doesn't mean the methodology is bad, just that it doesn't work in every case, and that some tweaking may be necessary.

PedroBatista 4y ago

> I think companies feel that if they aren’t going to “do DevOps”, that they need to rebrand the term “DevOps” to mean “what we happen to already do”. > To me, the most egregious example of this is when a company simply changes the job titles of all its ops people. John use to be an “IT Operations” guy, abut now he’s a “DevOps Engineer”. This is so true it hurts..

mekoka 4y ago

> We need better definitions for these terms When I point to the moon with my finger and say "moon", the fool looks at my finger and calls it the moon. Is it more important to mindlessly tag yourself "Agile", with all the bells and whistles adorning the capital A, or would you rather just be agile according to the sought out goals and philosophy, and adapted to your specific situation? Is it more important to have a clearer definition of "DevOps", so that you can more accurately implement all the prescribed workflows, or would you rather understand the pursued ideals, so that you can pick and choose what you need, according to your specific context, unencumbered by a slew of best, but ultimately useless, practices?

freshnode 4y ago

I think there is a third category - companies that have heard of DevOps, think it means that the development team and operations team talk to one another, which they already do (!) and shut down conversations after that. I've come across this archetype in several contracting gigs and I find it to be especially pernicious as it gives the incumbents (who usually have more weight in the org) ammo to effectively do nothing.

NonNefarious 4y ago

eweise 4y ago

I only do as much 'ops' work as required by the idiots in charge. Its much more productive for me to be working on business solutions than screw around with getting traces and logs into datadog and figuring out all the permission issues in AWS. Someone can specialize in that knowledge instead of everybody needing to know everything.

quantum_state 4y ago

Just being devil’s advocate, whatever the profession is, if someone keeps changing things, how would it reflect on his or her maturity in getting things done?

EricLeer 4y ago

I wonder how to use a devops flow for machine learning workloads. Something we are working on at my current job is how to balance the speed of releasing together with keeping data, pipeline and model versions aligned (where it for instance takes multiple days to release a new model). Releasing new features which need a model retraining is currently a very jarring experience.

ruh-roh 4y ago

While I generally agree about the annoyance with UI changes, and how that grinds on people, I hope we don't just handwave over the "CD for bug fixes is fine" bits. It varies product-to-product obviously, but I'd guess there is also a significant chunk of folks that are pleased that their functional annoyances/blockers (bugs) can be fixed quickly, without waiting for a monthly or quarterly release. These folks might not mind the fluctuating UIs, if it means they get the functional value they need asap.

not_me_ever 4y ago