6 ms·

How do you start learning this breadth of software engineering? I consider myself good in the python / django space, but where do I start with learning these in

How do you start learning this breadth of software engineering? I consider myself good in the python / django space, but where do I start with learning these infrastructure technologies? I find that I use them once or twice periodically, and then don't touch them for so long, so I forget much of what I have learned.

bombcar 6y ago
Document everything in excruciating detail - I go so far as to record all the commands I run; and when complete o destroy the machine and start again (or use a separate system) and verify that I accurately recorded every step. You can add additional text about why you did certain things - and then store the data in a wiki or checked into git or similar so you can find it when you need it.
- shoey 6y ago
  What is your system for keeping all of these notes. Do you just have a git repo for everything?
  - darekkay 6y ago
    Not OP, but I've been keeping a "journal" repo for the last 4 months. There's a single dev.md file in there, where I separate each entry with "---". Whenever I encounter an issue or learn something new, I document it for later reference.
  - RotaryTelephone 6y ago
    I do same and keep everything in Evernote organized in subfolders and tagged. Quick to find and helps 100% when it's time to rebuild/setup a server.
  - bombcar 6y ago
    It honestly depends on what it is - sometimes in the repository, for a wiki it’s stored in the wiki itself, otherwise it might be as simple as a text file in a web directory.
- greggman3 6y ago
  That's all great advice. What I find though is I don't do it enough for it not to change under me. Example: 1. Did a project on digital ocean, just ubuntu and node 2. Year later, Did a project using meteor, spent way too much time trying to get it all install with Vagrant (so all info from 1 was not useful) 3. Year later, Changed meteor setup to use docker ... so had to learn docker (so all info from 2 was not useful) 4. 2 Years later, Tried to do something with AWS lambda (so all info from 3 was not useful) 5. 1 Year later, Tried something with Apollo (so info from 4 was not useful) And to be honest, none of the projects' various needs are all that different. I feel like one "good" solution could have, should have, should now exist ... but I haven't found it. I guess I kind of feel like people who learned Rails back in the day found it met all their needs and they were able to do 50 projects on it. What is that thing today that if I learn today won't be out of date in 1-2yrs?
  - lupire 6y ago
    Just stop hopping. Anything youvet tried would be fine to stick with.
  - AlchemistCamp 6y ago
    Unless it's something bleeding edge or a a high churn JS stack, almost nothing should be out of date in just 1-2 years.
  - deleted 6y ago
    [deleted]
- tikkabhuna 6y ago
  I've used the script command and asciinema [1] before. Easy way to record steps without missing anything. [1] https://asciinema.org/ https://asciinema.org/
- Pokepokalypse 6y ago
  This. Most of the time, if you take the effort to document something, you'll retain it better anyway.
eandre 6y ago
The most natural way is to join a startup that is scaling. You can of course learn by doing it yourself on the side, but in practice "learning by doing" on the job is by far the most effective in my experience. I also hope we don't need to know all this stuff in the future. It's pretty really low-level and it's much better if we can focus more on creating differentiation and building your actual product. (Full disclosure: I've founded a startup that's trying to do exactly that, so I guess I'm biased!)
JimDabell 6y ago
It only takes two things – curiosity and time. When you are working on a project, if you hit the edge of your current knowledge / skills, push just a little bit further when it’s something that interests you instead of just aiming to hit the basic requirements / lean on other people. This minor effort compounds over time; do it for twenty years and you’ll be an expert in multiple disciplines and also an expert in how to tie them all together into one cohesive whole. Aim to be a “T-shaped” person, and just expand over time.
- test6554 6y ago
  I find it is best to do the tutorials. The really basic ones. First one tutorial or article, then another and another. Don't get distracted by using it on your own project yet. Do more and more tutorials. Read the docs. Not just the getting started guide. Read the docs for like 2 days. Then get a book and read that.
- MattyMc 6y ago
  It may also be helpful to share some details on effective ways to be curious. I’m a curious person too, but in the early days I just didn’t know where to start. My advice: - There is no defined learning path yet (to my knowledge). - Start by reading the GitHub readme of technology in these articles (ex: nginx or kubernetes). - If interested, try to spin up a tutorial app. - Try to make something useful. Maybe this is a spin on a tutorial, or something novel. This is the hardest but best way to learn. Finally, I’ll add that many folks learn these skills on the job either directly or having worked in proximity to new tech. It does seem this was how the author learned. Hope that helps! I’m sure others will have great advice, too!
- habosa 6y ago
  Actually I think it takes a third thing: need At least in my experience I can read about different architectures all day and sort of understand them, but I only really "get" it once I find a non-toy problem I need to solve and attempt to apply the knowledge. Then you see how it really works and form hard skills which stay with you.
  - 6y ago
bpicolo 6y ago
My strategy over the years has been to build a whole lot of otherwise useless side projects with incrementally different stacks, optimizing for doing it well with different technology instead of cobbling them together quickly.
moooo99 6y ago
For me, it's mostly learning by doing. At my day job as well as with my hobby projects. I initially started learning about the containerization and Kubernetes stuff mostly out of interest before I realized that there is a massive benefit to them, even during the development phase. I guess that's the reason why they are parts of most DevOps toolkits. While hobby projects can be a great start, the best way to learn is in a team of experience coworkers. The basic concepts of something like Kubernetes are very easy to grasp, leading people to believe Kubernetes is easy and completely missing the giant complexity the system introduces (that's way many people on here say its overpowered for 99% of projects, which I tend to agree on). Even with seemingly simple things like Docker, there is a massive amount of depth that's in my experience very hard to find in blog articles or YouTube tutorials. That being said, if you have to chance to learn about such things from your coworkers by applying them on your day job, I think the best choice is stell do have hobby/testing projects and combine the learning by doing aspect with some good books. I also recently learned about two YouTube channels that do a pretty good job with explaining such tools and applying them to the real world in a beginner friendly way. [1][2] [1] https://www.youtube.com/channel/UCdngmbVKX1Tgre699-XLlUA https://www.youtube.com/channel/UCdngmbVKX1Tgre699-XLlUA [2] https://www.youtube.com/user/Kamakazihoer https://www.youtube.com/user/Kamakazihoer
e3bc54b2 6y ago
I've wondered about this as well. I don't have a complete answer, but so far documenting things as I go about doing them helps, especially if I write down what I tried, what went wrong, what worked and why. Its a lot while starting off, but over time as the concepts sink and become habits, my docs move to higher abstractions automatically and then it is mostly clear. The key words for me are 'train of thought'. The solution (the how) is important obviously, and always useful when quickly referring, but when making bigger changes it is more important to remember th why. It is hella time consuming, needs dedication and practice and good tools (I couldn't start without org-mode myself)
- kaeland 6y ago
  I highly recommend org-mode for keeping track of anything dev-related.
smoe 6y ago
What has helped me most over the years is working in smaller companies, where you necessarily need to take on more responsabilities. My first job ~2005 was at a small shop with like 4-5 people and around 20 physical servers under our control and the same amount on-premise with clients (mix of Windows Servers, Linux distros and BSDs). We did have a sys admin person, but he was only responsible for the servers themselves and the base configuration. Everything application related running on it was our responsilibity as developers. And after that, in the following jobs and as a freelancer, there were I wide variety of things I had to ramp up on quickly. Different build processes, application monitoring, backups, different cloud providers, hidden costs, etc. Also I have been keeping a "Today I Learned" journal, where I just put small comments and snippets. It is hardly ever any deep insight, but for the most part "to do x in framework y solution z worked". It is also mostly a write-only journal. Just writing things down helps a lot with memory.
amzans 6y ago
Nothing like bringing down the entire production cluster and all services with it on a Friday afternoon due to a seemingly innocent "hotfix". Big learnings at the time, but now these make for good stories. I learned most of these tools at my day job through some catastrophic failures. From my experience, failure has always been the best teacher. Kubernetes just happens to be a great sandbox for failing hard :) Lots of stories here: https://k8s.af/ https://k8s.af/ However, I wouldn't reach for tools that didn't solve a problem I truly have, be it cost-effective scaling (my day job), or reusing what I already know best even if unconventional (my SaaS). I guess what I'm trying to say is: focus on solving your immediate problems first with the tools you already know. Your toolbelt will expand without you realizing it.
wayoutthere 6y ago
Work at a company that does DevOps well; this is a pretty common deployment pattern because it’s easy to test and gives a lot of flexibility. Many of the things OP is describing are only things you learn in a role with a production support component, which is where you find the tiny details like your health checks were inadequate and driving error rate spikes in certain scenarios without a specific config option, etc. Many of them are things that only pop up when you have some scale to deal with over a period of time, which makes it hard for hobbyists to pick up if they’re not on a team with prod support responsibilities.
throwaway823882 6y ago
Give yourself an ambitious pet project that requires you to learn and practice new things. Add to it over time so you continue to revisit the project with new requirements. And read all the docs, not just the parts you need to know.
codazoda 6y ago
> I find that I use them once or twice periodically, and then don't touch them for so long, so I forget much of what I have learned. This is why I "write". I started a decade ago capturing short notes for myself about the technologies I use. Writing it down helps me remember it in two ways. First, the act of writing (primarily by pen) is proven to increase your memory of a thing. Second, I can open my notes for step-by-step reminders. You don't have to blog publicly. Checkout the Zettelkasten method if you want to use Index cards. Keep a set of Markdown files in a private repo. Whatever floats your boat. If you keep notes in a notebook I found that labeling mine as "Stray Thoughts" was one of the best things for me. That prevents me from moving away for that notebook trying to categorize my thoughts. If they are just stray thoughts, I can put any random thought in that same notebook. The same thing works in a set of text files or a Zettel.
simonw 6y ago
For me the magic trick has always been side-projects. I have a lot of them, and each one is an opportunity to learn new tricks. (Over time I've learned that it's best to avoid side-projects which have user accounts and store data on behalf of other people, because that's not a side-project: it's an unpaid job.)
Arubis 6y ago
Addressing the implicit "am I doing career wrong?": there's folks that find working broadly across the whole stack really compelling (I am one of them) that will visibly have exposure to a _lot_ of stuff. Those folks (I am again one of them) are likely looking at your work and worriedly feeling as though they are Doing Career Wrong because they don't have your depth. Apologies if I'm merely projecting; my hope is that this is supportive.
adamhp 6y ago
Work somewhere where you're the big fish in a small pond. You're forced to wear a bunch of different hats and learn multiple tools across multiple business functions.
- Pokepokalypse 6y ago
  Also, sometimes, a great way to burn out quickly. . .
  - adamhp 5y ago
    Definitely. There's a sweet spot, where you're the go-to guy, and you leverage that for work-life balance (because no one else can do what you do), but it also means you're leaned on a lot to move things along. It definitely requires some fortitude and ability to manage time and expectations (and stick up for yourself).
fierro 6y ago
work on an infrastructure team. This is standard cloud infrastructure -- the networking (cloudflare/ingress LBs), k8s, terraform, CI/CD etc etc

bombcar 6y ago

Document everything in excruciating detail - I go so far as to record all the commands I run; and when complete o destroy the machine and start again (or use a separate system) and verify that I accurately recorded every step. You can add additional text about why you did certain things - and then store the data in a wiki or checked into git or similar so you can find it when you need it.

shoey 6y ago
What is your system for keeping all of these notes. Do you just have a git repo for everything?
- darekkay 6y ago
  Not OP, but I've been keeping a "journal" repo for the last 4 months. There's a single dev.md file in there, where I separate each entry with "---". Whenever I encounter an issue or learn something new, I document it for later reference.
- RotaryTelephone 6y ago
  I do same and keep everything in Evernote organized in subfolders and tagged. Quick to find and helps 100% when it's time to rebuild/setup a server.
- bombcar 6y ago
  It honestly depends on what it is - sometimes in the repository, for a wiki it’s stored in the wiki itself, otherwise it might be as simple as a text file in a web directory.
greggman3 6y ago
That's all great advice. What I find though is I don't do it enough for it not to change under me. Example: 1. Did a project on digital ocean, just ubuntu and node 2. Year later, Did a project using meteor, spent way too much time trying to get it all install with Vagrant (so all info from 1 was not useful) 3. Year later, Changed meteor setup to use docker ... so had to learn docker (so all info from 2 was not useful) 4. 2 Years later, Tried to do something with AWS lambda (so all info from 3 was not useful) 5. 1 Year later, Tried something with Apollo (so info from 4 was not useful) And to be honest, none of the projects' various needs are all that different. I feel like one "good" solution could have, should have, should now exist ... but I haven't found it. I guess I kind of feel like people who learned Rails back in the day found it met all their needs and they were able to do 50 projects on it. What is that thing today that if I learn today won't be out of date in 1-2yrs?
- lupire 6y ago
  Just stop hopping. Anything youvet tried would be fine to stick with.
- AlchemistCamp 6y ago
  Unless it's something bleeding edge or a a high churn JS stack, almost nothing should be out of date in just 1-2 years.
- deleted 6y ago
  [deleted]
tikkabhuna 6y ago
I've used the script command and asciinema [1] before. Easy way to record steps without missing anything. [1] https://asciinema.org/ https://asciinema.org/
Pokepokalypse 6y ago
This. Most of the time, if you take the effort to document something, you'll retain it better anyway.

eandre 6y ago

The most natural way is to join a startup that is scaling. You can of course learn by doing it yourself on the side, but in practice "learning by doing" on the job is by far the most effective in my experience. I also hope we don't need to know all this stuff in the future. It's pretty really low-level and it's much better if we can focus more on creating differentiation and building your actual product. (Full disclosure: I've founded a startup that's trying to do exactly that, so I guess I'm biased!)

JimDabell 6y ago

It only takes two things – curiosity and time. When you are working on a project, if you hit the edge of your current knowledge / skills, push just a little bit further when it’s something that interests you instead of just aiming to hit the basic requirements / lean on other people. This minor effort compounds over time; do it for twenty years and you’ll be an expert in multiple disciplines and also an expert in how to tie them all together into one cohesive whole. Aim to be a “T-shaped” person, and just expand over time.

test6554 6y ago
I find it is best to do the tutorials. The really basic ones. First one tutorial or article, then another and another. Don't get distracted by using it on your own project yet. Do more and more tutorials. Read the docs. Not just the getting started guide. Read the docs for like 2 days. Then get a book and read that.
MattyMc 6y ago
It may also be helpful to share some details on effective ways to be curious. I’m a curious person too, but in the early days I just didn’t know where to start. My advice: - There is no defined learning path yet (to my knowledge). - Start by reading the GitHub readme of technology in these articles (ex: nginx or kubernetes). - If interested, try to spin up a tutorial app. - Try to make something useful. Maybe this is a spin on a tutorial, or something novel. This is the hardest but best way to learn. Finally, I’ll add that many folks learn these skills on the job either directly or having worked in proximity to new tech. It does seem this was how the author learned. Hope that helps! I’m sure others will have great advice, too!
habosa 6y ago
Actually I think it takes a third thing: need At least in my experience I can read about different architectures all day and sort of understand them, but I only really "get" it once I find a non-toy problem I need to solve and attempt to apply the knowledge. Then you see how it really works and form hard skills which stay with you.
- 6y ago

bpicolo 6y ago

My strategy over the years has been to build a whole lot of otherwise useless side projects with incrementally different stacks, optimizing for doing it well with different technology instead of cobbling them together quickly.

moooo99 6y ago

For me, it's mostly learning by doing. At my day job as well as with my hobby projects. I initially started learning about the containerization and Kubernetes stuff mostly out of interest before I realized that there is a massive benefit to them, even during the development phase. I guess that's the reason why they are parts of most DevOps toolkits. While hobby projects can be a great start, the best way to learn is in a team of experience coworkers. The basic concepts of something like Kubernetes are very easy to grasp, leading people to believe Kubernetes is easy and completely missing the giant complexity the system introduces (that's way many people on here say its overpowered for 99% of projects, which I tend to agree on). Even with seemingly simple things like Docker, there is a massive amount of depth that's in my experience very hard to find in blog articles or YouTube tutorials. That being said, if you have to chance to learn about such things from your coworkers by applying them on your day job, I think the best choice is stell do have hobby/testing projects and combine the learning by doing aspect with some good books. I also recently learned about two YouTube channels that do a pretty good job with explaining such tools and applying them to the real world in a beginner friendly way. [1][2] [1] https://www.youtube.com/channel/UCdngmbVKX1Tgre699-XLlUA https://www.youtube.com/channel/UCdngmbVKX1Tgre699-XLlUA [2] https://www.youtube.com/user/Kamakazihoer https://www.youtube.com/user/Kamakazihoer

e3bc54b2 6y ago

I've wondered about this as well. I don't have a complete answer, but so far documenting things as I go about doing them helps, especially if I write down what I tried, what went wrong, what worked and why. Its a lot while starting off, but over time as the concepts sink and become habits, my docs move to higher abstractions automatically and then it is mostly clear. The key words for me are 'train of thought'. The solution (the how) is important obviously, and always useful when quickly referring, but when making bigger changes it is more important to remember th why. It is hella time consuming, needs dedication and practice and good tools (I couldn't start without org-mode myself)

kaeland 6y ago
I highly recommend org-mode for keeping track of anything dev-related.

smoe 6y ago

What has helped me most over the years is working in smaller companies, where you necessarily need to take on more responsabilities. My first job ~2005 was at a small shop with like 4-5 people and around 20 physical servers under our control and the same amount on-premise with clients (mix of Windows Servers, Linux distros and BSDs). We did have a sys admin person, but he was only responsible for the servers themselves and the base configuration. Everything application related running on it was our responsilibity as developers. And after that, in the following jobs and as a freelancer, there were I wide variety of things I had to ramp up on quickly. Different build processes, application monitoring, backups, different cloud providers, hidden costs, etc. Also I have been keeping a "Today I Learned" journal, where I just put small comments and snippets. It is hardly ever any deep insight, but for the most part "to do x in framework y solution z worked". It is also mostly a write-only journal. Just writing things down helps a lot with memory.

amzans 6y ago

Nothing like bringing down the entire production cluster and all services with it on a Friday afternoon due to a seemingly innocent "hotfix". Big learnings at the time, but now these make for good stories. I learned most of these tools at my day job through some catastrophic failures. From my experience, failure has always been the best teacher. Kubernetes just happens to be a great sandbox for failing hard :) Lots of stories here: https://k8s.af/ https://k8s.af/ However, I wouldn't reach for tools that didn't solve a problem I truly have, be it cost-effective scaling (my day job), or reusing what I already know best even if unconventional (my SaaS). I guess what I'm trying to say is: focus on solving your immediate problems first with the tools you already know. Your toolbelt will expand without you realizing it.

wayoutthere 6y ago

Work at a company that does DevOps well; this is a pretty common deployment pattern because it’s easy to test and gives a lot of flexibility. Many of the things OP is describing are only things you learn in a role with a production support component, which is where you find the tiny details like your health checks were inadequate and driving error rate spikes in certain scenarios without a specific config option, etc. Many of them are things that only pop up when you have some scale to deal with over a period of time, which makes it hard for hobbyists to pick up if they’re not on a team with prod support responsibilities.

throwaway823882 6y ago

Give yourself an ambitious pet project that requires you to learn and practice new things. Add to it over time so you continue to revisit the project with new requirements. And read all the docs, not just the parts you need to know.

codazoda 6y ago

> I find that I use them once or twice periodically, and then don't touch them for so long, so I forget much of what I have learned. This is why I "write". I started a decade ago capturing short notes for myself about the technologies I use. Writing it down helps me remember it in two ways. First, the act of writing (primarily by pen) is proven to increase your memory of a thing. Second, I can open my notes for step-by-step reminders. You don't have to blog publicly. Checkout the Zettelkasten method if you want to use Index cards. Keep a set of Markdown files in a private repo. Whatever floats your boat. If you keep notes in a notebook I found that labeling mine as "Stray Thoughts" was one of the best things for me. That prevents me from moving away for that notebook trying to categorize my thoughts. If they are just stray thoughts, I can put any random thought in that same notebook. The same thing works in a set of text files or a Zettel.

simonw 6y ago

For me the magic trick has always been side-projects. I have a lot of them, and each one is an opportunity to learn new tricks. (Over time I've learned that it's best to avoid side-projects which have user accounts and store data on behalf of other people, because that's not a side-project: it's an unpaid job.)

Arubis 6y ago

Addressing the implicit "am I doing career wrong?": there's folks that find working broadly across the whole stack really compelling (I am one of them) that will visibly have exposure to a _lot_ of stuff. Those folks (I am again one of them) are likely looking at your work and worriedly feeling as though they are Doing Career Wrong because they don't have your depth. Apologies if I'm merely projecting; my hope is that this is supportive.

adamhp 6y ago

Work somewhere where you're the big fish in a small pond. You're forced to wear a bunch of different hats and learn multiple tools across multiple business functions.

Pokepokalypse 6y ago
Also, sometimes, a great way to burn out quickly. . .
- adamhp 5y ago
  Definitely. There's a sweet spot, where you're the go-to guy, and you leverage that for work-life balance (because no one else can do what you do), but it also means you're leaned on a lot to move things along. It definitely requires some fortitude and ability to manage time and expectations (and stick up for yourself).

fierro 6y ago

work on an infrastructure team. This is standard cloud infrastructure -- the networking (cloudflare/ingress LBs), k8s, terraform, CI/CD etc etc