7 ms·

Can anyone offer a good guide to DevOps for people who don't directly use these tools but work with engineers who do and would like to learn more? The whole eco

by psv1 7y ago

Can anyone offer a good guide to DevOps for people who don't directly use these tools but work with engineers who do and would like to learn more? The whole ecosystem of servers, cloud infrastructure (and all of the different offerings there), Docker, Kubernetes, CICD tools etc is a bit overwhelming to get into.

marsrover 7y ago
I think the only way to learn them is to use them. You can do that locally or purchase a VPS and work there.
dsr_ 7y ago
Sure. None of the things you mentioned are DevOps. DevOps is two things: 1. Applying the methods of modern software development (version control, automation, DSLs...) to operations (provisioning, config, deployment, monitoring, backups...). 2. Reducing silo barriers between devs and ops groups so that everyone is working together as a team, rather than blaming each other for poor communication and the resulting messes. Then there are all the DevOps hijacking attempts, such as equating it to Agile or Scrum or XP, or insisting that it's a way to stop paying for expensive operations experts by making devs do it, or a way to stop paying for expensive devs by making ops do it, or a way to stop paying for expensive hardware by paying Amazon/Google/$CLOUD to do it. No matter what your software-as-a-service company actually does, it will need to execute certain things: - have computers to run software - have computers to develop software - have computers to run infrastructure support You can outsource various aspects of these things to different degrees. Anywhere you need computers, you have a choice of buying computers (and figuring out where to put them and how to run them and maintain them), or leasing computers (just a financing distinction), or renting existing computers (dedicated machines at a datacenter) or renting time on someone else's infrastructure. If you rent time, you can do so via virtual machines (which pretend to be whole servers) or containers (which pretend to be application deployments) or "serverless", which is actually a small auto-scaled container. Docker is a management scheme for containers. VMWare provides management schemes for virtual machines. Kubernetes is an extensive management scheme for virtual machines or containers. A continuous integration tool is, essentially, a program that notes that you have committed changes to your version control system and tries to build the resulting program. A continuous deployment system takes the CI's program and tries to put it into production (or, if you're sensible, into a QA deployment first).
- movedx 7y ago
  At last, someone who gets it. Absolutely nailed it. Great answer. I never log into my HN account anymore, but for this response I just had to say: yes. Well said. When you boil the Cloud, DevOps, CloudOps, SecOps, *Ops, CI, CD, Containers, VMs, and all the other technologies we've devised over the past ten years, you always end up at the basic building blocks. You eventually come to the conclusion that all we're really doing with all these new tools is adding software layers on top of those building blocks in an attempt to make them easier and faster to consume. And how have we done overall? Not bad, if you ask me. Some solutions are overkill for most people (K8s is an example of over kill for a start up and even an SME.) But Terraform, Ansible and GitLab (CI) are something I'm currently developing a highly opinionated video training course on because I believe they strike the right balance of improving on prior experiences without taking the absolute piss.
  - heurisko 7y ago
    I am a developer, who also dealt with ops in a small business context. I agree with Ansible striking a good balance between prior experience and the future of automating server configuration. I did a write-up on how I used it on my blog: https://heuristicservices.co.uk/2019/08/13/staging-and-production-servers-with-vagrant-and-ansible/ https://heuristicservices.co.uk/2019/08/13/staging-and-produ... The workflow worked really well, provisioning Vagrant servers in staging and Digital Ocean droplets in production.
    - movedx 7y ago
      Nice write up. Good job, mate. I moved away from Vagrant in favour of Terraform, but I agree Vagrant still holds its own and is a great choice (HashiCorp really nailed it, eh?)
    - GordonS 7y ago
      Thanks, I appreciated this blog post - I've struggled to get started with Ansible before, and this was just what I needed!
  - high_5 7y ago
nurettin 7y ago
After you accept that all these tools were built for overworked and stressed people who don't really have the time to learn things deeply, it becomes much easier. In fact, most of the programming ecosystem and systems administration works like this. Try focusing on "what do I want?", get a superficial understanding on how the tool works, then try to apply that knowledge to your search engine query. For example: Say you know that docker has images and containers. That means it is somehow going to install an operating system into your operating system and make an image. Then you will copy your program into that image. Then you will start a container (an instance) based on that image. And this is basically all you need to know about docker to start searching for how to do things. Like "how to build a docker image?" or "how to start a docker container?". Another example. You know that Integration Testing means running your servers and running tests against them as if they are in actual production and continuous integration is a service that runs your integration tests everytime someone merges a branch to a monitored branch in a version control system. From here on, you are able to look up how to set a monitored branch, how to create a build machine and how to scale it.
- elevenoh 7y ago
  This is how you end up with low-staying-value understanding built on shaky, fragile footing.
  - aldoushuxley001 7y ago
    Debatable, I think this sort of investigation is what leads to a more robust understanding compared to some abstract, non-applied information that has dubious relevance st the moment.
  - owenmarshall 7y ago
    "And what the heck do you call an act like that?" "I call it 'modern software development practices!'"
  - noxToken 7y ago
    You can but not necessarily will. It can be valuable (to an extent) to abstract certain parts of process away from new users or employees to aid in bootstrapping. That's the message that I get from GP's comment. What these tools can't do is replace experts that can get between the commands to diagnose the root cause of an issue rather than the taking stabs at the dark of a cryptic error message. These tools for DevOps are no different than the tools and tutorials for developers. It's fine to copy-paste a tutorial that launches an entire webapp from scratch just to get your project off the ground, but if you don't eventually learn what that 10-minute autorun.sh script is doing behind the scenes, you'll fall behind.
swyx 7y ago
nice guide to devops linked at the bottom of the article https://www.techrepublic.com/article/devops-the-smart-persons-guide/ https://www.techrepublic.com/article/devops-the-smart-person...
movedx 7y ago
I'm currently in the process of developing a video training course that teaches Terraform, Ansible, Packer and GitLab (CI) as a set of interwoven, dependent tools. Is this something you feel would scratch your itch? Would you be willing to have a quick chat? I'd pay for your time, of course. I need to gather feed back from people looking to develop their skills into the CloudOps space and understand what it is they're looking for.
- programmertote 7y ago
  I have been looking for a entry point to start learning containerization and other fancy, related tools that I've been reading on HN in the past couple of years. I mostly write backend code/scripts, so I'm very new to dev ops stuff. The immediate need for me is to package up a C#/.NET web app and its components (DB, etc.) into a container so that I can deploy it on any big-name cloud provider (Azure, AWS, Google Cloud). Now after reading through the comments in this HN post, I am not sure if I should choose Docker or something else. If you have any suggestion, I would love to learn. I'm more than happy to provide you with feedback and such (even for free) if I can learn from your tutorials. Thank you.
  - aeorgnoieang 7y ago
    I'd suggest starting with just scripts, e.g. Bash, PowerShell, anything – that's assuming you have automated builds for both your web app and its components first. (So, automate builds of your app and all of its components first if you don't already have that.) Pick one cloud provider first and write some setup scripts, i.e. scripts to build an initial (minimal) environment, 'from scratch', for your app and its components, e.g. create a new EC2 instance for the web app, upload your build package to it, etc.. Write the scripts so that any 'secrets', e.g. your AWS API key, are provided as either environment variables or regular command line arguments. Then, still for the first cloud provider, write some update scripts, i.e. scripts to update your web app and its components. Assuming your app, or its deployed 'instances', are relatively small and intended to serve a modest load, I'd suggest starting out treating the cloud servers or services as 'pets', i.e. entities you distinguish by name and for which you would be 'sad' if they 'died' (crashed or shutdown). At larger scales, it's often worth treating servers as 'cattle', i.e. a mass of nameless entities, but you probably won't need that at this point or anytime soon. (You'll know better tho.) As for "containerization and other fancy, related tools that I've been reading on HN in the past couple of years", they're just like any other software – tools that can be used but aren't ever strictly necessary. Containers are, basically, virtualized OSes, and they can be (very) useful. In my opinion, they're most useful as a way to bundle components of an app with an OS (and other OS components). That you can run those containers in different environments and be reasonably assured they're (mostly) identical can be a big benefit. But there are associated costs too (as with anything)! But if your app and all of its components can comfortably fit on a single (virtual) server, the cost of any changes you need to make to your app to run inside them are probably not worth the (currently) modest benefits. And all of the other "fancy ... tools" are generally even more a matter of tradeoffs you'll need to make. Once your app, or (production) instances of it, are distributed over several, or many, servers, and you start adding things like load balancers, caching, separate search services, etc., then the benefits of the other 'fancy' tools will start to make more sense. But, like with many things, it's good experience to directly run into some of the issues that containers and the other fancy tools aim to solve, and really try to solve them yourself with DIY solutions, before committing to use yet another program or tool to do it for you. Of course, if you just want to learn those tools, and you'd like to use your own app as a 'motivate example', that's perfectly reasonable and valid too. I would recommend tho not to lean on those tools for your own immediate needs unless you really need them.
orthoxerox 7y ago
What most enterprises think is DevOps is different from what dsr_ wrote. They have admins that maintain their pool of servers. They have developers that are fluent in the stack of their application. They decide they need to have some of that cloud, containers, CI/CD stuff. Turns out they need people who can write code that builds their programs, tests their programs, packages their programs, provisions cloud infrastructure, sets up that infrastructure, deploys the packaged program on their infrastructure and finally monitors its health and performance. Most of their admins say they are not programmers, so it's not their job. Most of their programmers say they are there to write Java/C#/Python/JS, so it's not their job either. They find some people who don't mind learning all these things and call them their DevOps team. In a not perfect, but generally just world this team disseminates their knowledge across both programmers and admins, making both aware of each other. Programmers now think about the infrastructure they need, admins now think about the workloads their infrastructure runs. In an unjust world, you end up with three silos. Programmers say their code compiles on their machine, admins say they have installed the new server, devops frantically try to build some pipeline that deploys that code on that server.
- StreamBright 7y ago
  >> What most enterprises think You mean the companies we regularly fail every aspect of engineering? dsr_ summed up pretty well how Amazon and a bunch of companies think abut DevOps. Coincidentally these companies produce the highest grade of software, tools, services etc. >> Most of their admins say they are not programmers, so it's not their job. What you are describing is the 90s approach to IT. These companies disappear really fast. IT is changing just like agriculture was changing long time ago. Toffler talks about this in The Third Wave. Old approach: lets do everything by hand New approach: automate most of the things you can >> In an unjust world, you end up with three silos. Programmers say their code compiles on their machine, admins say they have installed the new server, devops frantically try to build some pipeline that deploys that code on that server. I migrated countless companies from 90s approach to CI/CD world, they never looked back. You just think that because there are late adopters this world is going to exist indefinitely. I do not think so.
  - 7y ago

by psv1 7y ago

marsrover 7y ago

I think the only way to learn them is to use them. You can do that locally or purchase a VPS and work there.

dsr_ 7y ago

Sure. None of the things you mentioned are DevOps. DevOps is two things: 1. Applying the methods of modern software development (version control, automation, DSLs...) to operations (provisioning, config, deployment, monitoring, backups...). 2. Reducing silo barriers between devs and ops groups so that everyone is working together as a team, rather than blaming each other for poor communication and the resulting messes. Then there are all the DevOps hijacking attempts, such as equating it to Agile or Scrum or XP, or insisting that it's a way to stop paying for expensive operations experts by making devs do it, or a way to stop paying for expensive devs by making ops do it, or a way to stop paying for expensive hardware by paying Amazon/Google/$CLOUD to do it. No matter what your software-as-a-service company actually does, it will need to execute certain things: - have computers to run software - have computers to develop software - have computers to run infrastructure support You can outsource various aspects of these things to different degrees. Anywhere you need computers, you have a choice of buying computers (and figuring out where to put them and how to run them and maintain them), or leasing computers (just a financing distinction), or renting existing computers (dedicated machines at a datacenter) or renting time on someone else's infrastructure. If you rent time, you can do so via virtual machines (which pretend to be whole servers) or containers (which pretend to be application deployments) or "serverless", which is actually a small auto-scaled container. Docker is a management scheme for containers. VMWare provides management schemes for virtual machines. Kubernetes is an extensive management scheme for virtual machines or containers. A continuous integration tool is, essentially, a program that notes that you have committed changes to your version control system and tries to build the resulting program. A continuous deployment system takes the CI's program and tries to put it into production (or, if you're sensible, into a QA deployment first).

movedx 7y ago
At last, someone who gets it. Absolutely nailed it. Great answer. I never log into my HN account anymore, but for this response I just had to say: yes. Well said. When you boil the Cloud, DevOps, CloudOps, SecOps, *Ops, CI, CD, Containers, VMs, and all the other technologies we've devised over the past ten years, you always end up at the basic building blocks. You eventually come to the conclusion that all we're really doing with all these new tools is adding software layers on top of those building blocks in an attempt to make them easier and faster to consume. And how have we done overall? Not bad, if you ask me. Some solutions are overkill for most people (K8s is an example of over kill for a start up and even an SME.) But Terraform, Ansible and GitLab (CI) are something I'm currently developing a highly opinionated video training course on because I believe they strike the right balance of improving on prior experiences without taking the absolute piss.
- heurisko 7y ago
  I am a developer, who also dealt with ops in a small business context. I agree with Ansible striking a good balance between prior experience and the future of automating server configuration. I did a write-up on how I used it on my blog: https://heuristicservices.co.uk/2019/08/13/staging-and-production-servers-with-vagrant-and-ansible/ https://heuristicservices.co.uk/2019/08/13/staging-and-produ... The workflow worked really well, provisioning Vagrant servers in staging and Digital Ocean droplets in production.
  - movedx 7y ago
    Nice write up. Good job, mate. I moved away from Vagrant in favour of Terraform, but I agree Vagrant still holds its own and is a great choice (HashiCorp really nailed it, eh?)
  - GordonS 7y ago
    Thanks, I appreciated this blog post - I've struggled to get started with Ansible before, and this was just what I needed!
- high_5 7y ago

nurettin 7y ago

After you accept that all these tools were built for overworked and stressed people who don't really have the time to learn things deeply, it becomes much easier. In fact, most of the programming ecosystem and systems administration works like this. Try focusing on "what do I want?", get a superficial understanding on how the tool works, then try to apply that knowledge to your search engine query. For example: Say you know that docker has images and containers. That means it is somehow going to install an operating system into your operating system and make an image. Then you will copy your program into that image. Then you will start a container (an instance) based on that image. And this is basically all you need to know about docker to start searching for how to do things. Like "how to build a docker image?" or "how to start a docker container?". Another example. You know that Integration Testing means running your servers and running tests against them as if they are in actual production and continuous integration is a service that runs your integration tests everytime someone merges a branch to a monitored branch in a version control system. From here on, you are able to look up how to set a monitored branch, how to create a build machine and how to scale it.

elevenoh 7y ago
This is how you end up with low-staying-value understanding built on shaky, fragile footing.
- aldoushuxley001 7y ago
  Debatable, I think this sort of investigation is what leads to a more robust understanding compared to some abstract, non-applied information that has dubious relevance st the moment.
- owenmarshall 7y ago
  "And what the heck do you call an act like that?" "I call it 'modern software development practices!'"
- noxToken 7y ago
  You can but not necessarily will. It can be valuable (to an extent) to abstract certain parts of process away from new users or employees to aid in bootstrapping. That's the message that I get from GP's comment. What these tools can't do is replace experts that can get between the commands to diagnose the root cause of an issue rather than the taking stabs at the dark of a cryptic error message. These tools for DevOps are no different than the tools and tutorials for developers. It's fine to copy-paste a tutorial that launches an entire webapp from scratch just to get your project off the ground, but if you don't eventually learn what that 10-minute autorun.sh script is doing behind the scenes, you'll fall behind.

swyx 7y ago

nice guide to devops linked at the bottom of the article https://www.techrepublic.com/article/devops-the-smart-persons-guide/ https://www.techrepublic.com/article/devops-the-smart-person...

movedx 7y ago

I'm currently in the process of developing a video training course that teaches Terraform, Ansible, Packer and GitLab (CI) as a set of interwoven, dependent tools. Is this something you feel would scratch your itch? Would you be willing to have a quick chat? I'd pay for your time, of course. I need to gather feed back from people looking to develop their skills into the CloudOps space and understand what it is they're looking for.

programmertote 7y ago
I have been looking for a entry point to start learning containerization and other fancy, related tools that I've been reading on HN in the past couple of years. I mostly write backend code/scripts, so I'm very new to dev ops stuff. The immediate need for me is to package up a C#/.NET web app and its components (DB, etc.) into a container so that I can deploy it on any big-name cloud provider (Azure, AWS, Google Cloud). Now after reading through the comments in this HN post, I am not sure if I should choose Docker or something else. If you have any suggestion, I would love to learn. I'm more than happy to provide you with feedback and such (even for free) if I can learn from your tutorials. Thank you.
- aeorgnoieang 7y ago
  I'd suggest starting with just scripts, e.g. Bash, PowerShell, anything – that's assuming you have automated builds for both your web app and its components first. (So, automate builds of your app and all of its components first if you don't already have that.) Pick one cloud provider first and write some setup scripts, i.e. scripts to build an initial (minimal) environment, 'from scratch', for your app and its components, e.g. create a new EC2 instance for the web app, upload your build package to it, etc.. Write the scripts so that any 'secrets', e.g. your AWS API key, are provided as either environment variables or regular command line arguments. Then, still for the first cloud provider, write some update scripts, i.e. scripts to update your web app and its components. Assuming your app, or its deployed 'instances', are relatively small and intended to serve a modest load, I'd suggest starting out treating the cloud servers or services as 'pets', i.e. entities you distinguish by name and for which you would be 'sad' if they 'died' (crashed or shutdown). At larger scales, it's often worth treating servers as 'cattle', i.e. a mass of nameless entities, but you probably won't need that at this point or anytime soon. (You'll know better tho.) As for "containerization and other fancy, related tools that I've been reading on HN in the past couple of years", they're just like any other software – tools that can be used but aren't ever strictly necessary. Containers are, basically, virtualized OSes, and they can be (very) useful. In my opinion, they're most useful as a way to bundle components of an app with an OS (and other OS components). That you can run those containers in different environments and be reasonably assured they're (mostly) identical can be a big benefit. But there are associated costs too (as with anything)! But if your app and all of its components can comfortably fit on a single (virtual) server, the cost of any changes you need to make to your app to run inside them are probably not worth the (currently) modest benefits. And all of the other "fancy ... tools" are generally even more a matter of tradeoffs you'll need to make. Once your app, or (production) instances of it, are distributed over several, or many, servers, and you start adding things like load balancers, caching, separate search services, etc., then the benefits of the other 'fancy' tools will start to make more sense. But, like with many things, it's good experience to directly run into some of the issues that containers and the other fancy tools aim to solve, and really try to solve them yourself with DIY solutions, before committing to use yet another program or tool to do it for you. Of course, if you just want to learn those tools, and you'd like to use your own app as a 'motivate example', that's perfectly reasonable and valid too. I would recommend tho not to lean on those tools for your own immediate needs unless you really need them.

orthoxerox 7y ago

What most enterprises think is DevOps is different from what dsr_ wrote. They have admins that maintain their pool of servers. They have developers that are fluent in the stack of their application. They decide they need to have some of that cloud, containers, CI/CD stuff. Turns out they need people who can write code that builds their programs, tests their programs, packages their programs, provisions cloud infrastructure, sets up that infrastructure, deploys the packaged program on their infrastructure and finally monitors its health and performance. Most of their admins say they are not programmers, so it's not their job. Most of their programmers say they are there to write Java/C#/Python/JS, so it's not their job either. They find some people who don't mind learning all these things and call them their DevOps team. In a not perfect, but generally just world this team disseminates their knowledge across both programmers and admins, making both aware of each other. Programmers now think about the infrastructure they need, admins now think about the workloads their infrastructure runs. In an unjust world, you end up with three silos. Programmers say their code compiles on their machine, admins say they have installed the new server, devops frantically try to build some pipeline that deploys that code on that server.

StreamBright 7y ago
>> What most enterprises think You mean the companies we regularly fail every aspect of engineering? dsr_ summed up pretty well how Amazon and a bunch of companies think abut DevOps. Coincidentally these companies produce the highest grade of software, tools, services etc. >> Most of their admins say they are not programmers, so it's not their job. What you are describing is the 90s approach to IT. These companies disappear really fast. IT is changing just like agriculture was changing long time ago. Toffler talks about this in The Third Wave. Old approach: lets do everything by hand New approach: automate most of the things you can >> In an unjust world, you end up with three silos. Programmers say their code compiles on their machine, admins say they have installed the new server, devops frantically try to build some pipeline that deploys that code on that server. I migrated countless companies from 90s approach to CI/CD world, they never looked back. You just think that because there are late adopters this world is going to exist indefinitely. I do not think so.
- 7y ago