6 ms·

All of the points are very valid and valuable. But by reading only this post, you will have basically no idea how to go ahead and do all these things. You have

All of the points are very valid and valuable. But by reading only this post, you will have basically no idea how to go ahead and do all these things. You have to scour the internet, and collect every blog post and video ever produced on the subject, and intone them all, and start experimenting, and just sort of create your own version of what these things are. I'm going to interviews right now where people are trying to hire a "Senior DevOps Engineer". They seem to think hiring this somewhat low-level employee will magically imbue their organization with DevOps principles, and suddenly everything will be cheaper and faster. I tell them it may take them 2 years to build a solid DevOps practice, assuming they do everything right, and they frown. Why can't we just have the DevOps right now? To that I reply, go hire a company that specializes in DevOps. And again they frown. I don't think it should be this hard. I think we can consolidate all the important knowledge and pre-packaged work, collaboratively, to empower teams to actually get shit done quicker and better. I think we can make it so nobody ever has to go through the painful process of learning by trial and error ever again. And, of course, I think it should all be free. So I've started a wiki, and I hope I can convince a few more people to help contribute content. GruntWork, I love what you're doing, but I'm also hoping to make your business obsolete. :)

perfunctory 8y ago
> I don't think it should be this hard. Totally agree. Unfortunately, you are up against the industry that is not interested in getting shit done quicker and better. "Institutions will try to preserve the problem to which they are the solution" https://en.wikipedia.org/wiki/Clay_Shirky#Shirky_principle https://en.wikipedia.org/wiki/Clay_Shirky#Shirky_principle
- therealdrag0 8y ago
  Couldn't believe my ears when I heard an Ops guy say "we can't automate stuff, then what would we do?" He came around after he had 3x more work then he could ever do himself.
auslander 8y ago
> we can consolidate all the important knowledge and pre-packaged work Noble intent, but time for that is not yet. The state of cloud infrastructure practices today is a primordial soup of ideas, marketing and the myriad of tools trying to survive on the market, like Terraform vs Ansible vs Cloudformation :) Good infrastructure is hard, there is global networking with CDNs, DNS, CI/CD, Git integration, HA, monitoring, security, you name it. Thats why you will not find the code on Github, only blog posts on Medium, code brings bread and butter to so many people, including me :), why sharing it ? And most important. To create really reusable by many infra code is even harder. You need to feel what features should be there for everyone, and what is bloat. It'll take many times the effort of writing non-reusable architecture for one company. > .. And again they frown Made me smile :)
jasode 8y ago
>I think we can consolidate all the important knowledge and pre-packaged work, [...]. I think we can make it so nobody ever has to go through the painful process of learning by trial and error ever again. Just an fyi... that goal is only possible for ideas that have stabilized and broad consensus has converged into widely accepted best practices. Yesterday's "hacks" that are insanely complex get vetted (or modified and replaced) and eventually get folded into the baseline of Normal Things Everybody Knows. But... the world isn't standing still and it evolves in complexity. Therefore, there's always new complexity that requires new hacks that everybody reinvents. At a high level, the cycle looks like: chaos --> create order --> new chaos again We see a chaos of hacks and complexity. We then notice recurring patterns in the disparate hacks and extract some unifying principle that can "simplify" the complexity. If we stop here, it seems we've tamed disorder and everything should be able to be documented and people can stop wasting time repeating everyone else's past mistakes. But all we've really done is raise the baseline of systems sophistication. There's always a new delta of chaos and complexity. E.g. In the 1980s, sysadmins might implement a rudimentary sync of userids with a shell script and copying "/etc/passwd" files around. But we notice that's chaotic and stupid and doesn't scale. So we rationalize the system with a more unified technology such as a LDAP identity server. But utopia is still not achieved because while LDAP might be ok for employees' sign-on accounts, it's not suitable for millions of customer accounts of a B2C web business. The complexity treadmill is neverending! >GruntWork, I love what you're doing, but I'm also hoping to make your business obsolete. If companies like GruntWork and AWS are properly doing their job, they will never be obsolete because the changing world keeps throwing new demands at us and this constant delta of new chaos is always there that needs to be "solved".
- deleted 8y ago
  [deleted]
- peterwwillis 8y ago
  Well sure, but it's not like they're in a vacuum. Anyone working in the industry sees the changes. We all could use a place to share information, and a place to discuss and remember over time. I think a forum (or mailing list?) could be used to manage the chaos, and its conversations used to help govern the order of the Wiki. Not that Wikis need order... It just sort of emerges over time.
mmillin 8y ago
> So I've started a wiki, and I hope I can convince a few more people to help contribute content. Where can this be found?
- peterwwillis 8y ago
  Right now, DevOps.yoga (and SRE.pizza). Shamefully little content so far. If you'd like to add something, just send a PR and I'll merge it!
solatic 8y ago
> I don't think it should be this hard The difficulty arises from the underlying complexity. The underlying layers are necessarily complex, and the law of conservation of complexity applies. Documentation can be improved, and it will improve with time - and managed services will get more powerful. But yeah.
isodude 8y ago
Would you please relate to Cloud native then? https://landscape.cncf.io/format=landscape https://landscape.cncf.io/format=landscape Took me a while before I found that, would have been helpful earlier on.
devonkim 8y ago
The fundamental truth I've seen is that DevOps (the practices, not the role) is anti-Taylorist organizational design when most corporations are run completely opposite to that. The successes happen only when from top-down the organization has come to the sober realization that they really need to break down barriers and enable individuals in addition to hiring those capable of working across multiple functions (most people used to being coddled with purely focused work and using managers as shields and go-betweens don't work out well in this transition). Most organizations only want to give people just enough to be productive to a certain capacity not just because of least access principles in security but also because of the cargo cult of manager worship that is antithetical to both Agile and DevOps. Unfortunately, I've also seen plenty of consulting firms selling "Devops Solutions" and productizing everything because their customers look at everything as products and solutions that they acquire and presume that their culture is fine as-is. While I don't doubt that these firms make plenty of money I don't think the practice will be viable long-term because the problems are entirely due to culture rather than tools or even people.
- jammygit 8y ago
  Pardon my ignorance, but I thought Taylorism was primarily about measurement, & standardization of work practices. Could you elaborate on how it views organizational structures? Or do you just mean a mentality where managers end up viewing themselves as scientists and then viewing their workers as various dials to turn?
  - devonkim 8y ago
    Sure, Taylorism is indeed about measurements and standardization as well but where it differs greatly from the approach that Deming espoused is in where authority and primary allocators of resources are positioned and housed. In Taylorism, it’s much more centralized and managers are the dedicated resources that decide allocation of resources primarily and need information constantly to make decisions (hence a culture of most corporations constantly asking for reports that are usually outdated / irrelevant by the time corrective actions are enacted). In the approaches toward quality that Deming talked about, much more emphasis is placed upon the end product and responsibilities and authority being granted closer to the work being performed (this is not Communism - quite the opposite!). The infamously centralized US Army has started to abandon many Taylorist principles to help fight in the Middle East against a decentralized enemy force. In a really well done “devops” culture while managers are still valuable, their roles are a lot more limited in scope day-to-day than in traditionally structured organizations. Other approaches include “flat” organizations (fully peer-to-peer network mesh) and holocracy. This, in Deming’s view there are many less managers and you may not need them at the smallest unit of organizational structure (you may report to someone as a formality but you have more autonomy and KPIs attached to you). Deming’s work was not widely accepted in the US, but his approach was accepted at a place many Americans do recognize for superior quality and consistency - Toyota. Every other devops process of feedback and organizational structure is derived squarely from Deming’s principles. I always thought it’s rather ironic that a society known for being relentlessly individualist adopted corporate practices of conformity and a society known for being conformist took on an organizational philosophy that put more control into individuals. Maybe it was done this way to counter natural social inclinations in wider society? I’m not sure honestly. My gut feeling is that Taylorism is precisely how large militaries historically work and following WW2 this was easier for American workers to adopt.

perfunctory 8y ago

> I don't think it should be this hard. Totally agree. Unfortunately, you are up against the industry that is not interested in getting shit done quicker and better. "Institutions will try to preserve the problem to which they are the solution" https://en.wikipedia.org/wiki/Clay_Shirky#Shirky_principle https://en.wikipedia.org/wiki/Clay_Shirky#Shirky_principle

therealdrag0 8y ago
Couldn't believe my ears when I heard an Ops guy say "we can't automate stuff, then what would we do?" He came around after he had 3x more work then he could ever do himself.

auslander 8y ago

> we can consolidate all the important knowledge and pre-packaged work Noble intent, but time for that is not yet. The state of cloud infrastructure practices today is a primordial soup of ideas, marketing and the myriad of tools trying to survive on the market, like Terraform vs Ansible vs Cloudformation :) Good infrastructure is hard, there is global networking with CDNs, DNS, CI/CD, Git integration, HA, monitoring, security, you name it. Thats why you will not find the code on Github, only blog posts on Medium, code brings bread and butter to so many people, including me :), why sharing it ? And most important. To create really reusable by many infra code is even harder. You need to feel what features should be there for everyone, and what is bloat. It'll take many times the effort of writing non-reusable architecture for one company. > .. And again they frown Made me smile :)

jasode 8y ago

>I think we can consolidate all the important knowledge and pre-packaged work, [...]. I think we can make it so nobody ever has to go through the painful process of learning by trial and error ever again. Just an fyi... that goal is only possible for ideas that have stabilized and broad consensus has converged into widely accepted best practices. Yesterday's "hacks" that are insanely complex get vetted (or modified and replaced) and eventually get folded into the baseline of Normal Things Everybody Knows. But... the world isn't standing still and it evolves in complexity. Therefore, there's always new complexity that requires new hacks that everybody reinvents. At a high level, the cycle looks like: chaos --> create order --> new chaos again We see a chaos of hacks and complexity. We then notice recurring patterns in the disparate hacks and extract some unifying principle that can "simplify" the complexity. If we stop here, it seems we've tamed disorder and everything should be able to be documented and people can stop wasting time repeating everyone else's past mistakes. But all we've really done is raise the baseline of systems sophistication. There's always a new delta of chaos and complexity. E.g. In the 1980s, sysadmins might implement a rudimentary sync of userids with a shell script and copying "/etc/passwd" files around. But we notice that's chaotic and stupid and doesn't scale. So we rationalize the system with a more unified technology such as a LDAP identity server. But utopia is still not achieved because while LDAP might be ok for employees' sign-on accounts, it's not suitable for millions of customer accounts of a B2C web business. The complexity treadmill is neverending! >GruntWork, I love what you're doing, but I'm also hoping to make your business obsolete. If companies like GruntWork and AWS are properly doing their job, they will never be obsolete because the changing world keeps throwing new demands at us and this constant delta of new chaos is always there that needs to be "solved".

deleted 8y ago
[deleted]
peterwwillis 8y ago
Well sure, but it's not like they're in a vacuum. Anyone working in the industry sees the changes. We all could use a place to share information, and a place to discuss and remember over time. I think a forum (or mailing list?) could be used to manage the chaos, and its conversations used to help govern the order of the Wiki. Not that Wikis need order... It just sort of emerges over time.

mmillin 8y ago

> So I've started a wiki, and I hope I can convince a few more people to help contribute content. Where can this be found?

peterwwillis 8y ago
Right now, DevOps.yoga (and SRE.pizza). Shamefully little content so far. If you'd like to add something, just send a PR and I'll merge it!

solatic 8y ago

> I don't think it should be this hard The difficulty arises from the underlying complexity. The underlying layers are necessarily complex, and the law of conservation of complexity applies. Documentation can be improved, and it will improve with time - and managed services will get more powerful. But yeah.

isodude 8y ago

Would you please relate to Cloud native then? https://landscape.cncf.io/format=landscape https://landscape.cncf.io/format=landscape Took me a while before I found that, would have been helpful earlier on.

devonkim 8y ago

The fundamental truth I've seen is that DevOps (the practices, not the role) is anti-Taylorist organizational design when most corporations are run completely opposite to that. The successes happen only when from top-down the organization has come to the sober realization that they really need to break down barriers and enable individuals in addition to hiring those capable of working across multiple functions (most people used to being coddled with purely focused work and using managers as shields and go-betweens don't work out well in this transition). Most organizations only want to give people just enough to be productive to a certain capacity not just because of least access principles in security but also because of the cargo cult of manager worship that is antithetical to both Agile and DevOps. Unfortunately, I've also seen plenty of consulting firms selling "Devops Solutions" and productizing everything because their customers look at everything as products and solutions that they acquire and presume that their culture is fine as-is. While I don't doubt that these firms make plenty of money I don't think the practice will be viable long-term because the problems are entirely due to culture rather than tools or even people.

jammygit 8y ago
Pardon my ignorance, but I thought Taylorism was primarily about measurement, & standardization of work practices. Could you elaborate on how it views organizational structures? Or do you just mean a mentality where managers end up viewing themselves as scientists and then viewing their workers as various dials to turn?
- devonkim 8y ago
  Sure, Taylorism is indeed about measurements and standardization as well but where it differs greatly from the approach that Deming espoused is in where authority and primary allocators of resources are positioned and housed. In Taylorism, it’s much more centralized and managers are the dedicated resources that decide allocation of resources primarily and need information constantly to make decisions (hence a culture of most corporations constantly asking for reports that are usually outdated / irrelevant by the time corrective actions are enacted). In the approaches toward quality that Deming talked about, much more emphasis is placed upon the end product and responsibilities and authority being granted closer to the work being performed (this is not Communism - quite the opposite!). The infamously centralized US Army has started to abandon many Taylorist principles to help fight in the Middle East against a decentralized enemy force. In a really well done “devops” culture while managers are still valuable, their roles are a lot more limited in scope day-to-day than in traditionally structured organizations. Other approaches include “flat” organizations (fully peer-to-peer network mesh) and holocracy. This, in Deming’s view there are many less managers and you may not need them at the smallest unit of organizational structure (you may report to someone as a formality but you have more autonomy and KPIs attached to you). Deming’s work was not widely accepted in the US, but his approach was accepted at a place many Americans do recognize for superior quality and consistency - Toyota. Every other devops process of feedback and organizational structure is derived squarely from Deming’s principles. I always thought it’s rather ironic that a society known for being relentlessly individualist adopted corporate practices of conformity and a society known for being conformist took on an organizational philosophy that put more control into individuals. Maybe it was done this way to counter natural social inclinations in wider society? I’m not sure honestly. My gut feeling is that Taylorism is precisely how large militaries historically work and following WW2 this was easier for American workers to adopt.