4 ms·
You learn large scale system design via real world application, being in the room when it happens, and from closely working with folks who are already practicin
by danrl 4y ago
You learn large scale system design via real world application, being in the room when it happens, and from closely working with folks who are already practicing it. This kind of work is sometimes as much art as it is science, highly specific to the situation at hand and requirements of the system to be designed.
It absolutely bothers me that the resources out there are few and far between. It is hard to change that. I encountered occasionally resistance in organizations to share more openly the process, e.g. I suggested video taping it and uploading the video as a start. Naturally, plenty of confidential information can be disclosed in such design sessions, so enthusiasm about sharing recordings is rather low, even when everyone in the room agreed to be taped in the first place (which in itself is almost impossible).
I am bothered by engineering topics that have this „tribalism“ way of education. In networking one of these areas is BGP. No one lets you near Border Routing until you can do it perfectly, because of it potentially catastrophic impact (Pepperidge Farms remembers Pakistan Telecom hijacking YouTube!). When I tried to solve that one I ended up spinning up a whole ISP which’s sole purpose is to play with and break BGP. [1]
For system design I haven’t found a solution yet, ideas welcome. In the meantime, all you can do is read the little that is available (e.g. [2]) and try to talk to as many practitioners as possible, asking them to share or simulate a session privately.
1: https://www.nonattached.net https://www.nonattached.net
2: https://cloud.google.com/blog/products/management-tools/sre-principles-and-flashcards-to-design-nalsd https://cloud.google.com/blog/products/management-tools/sre-...
- kodah 4y agoI'll second that the best way to get better is to be a practitioner. Part of the reason that there's not more material on systems design is because the way we design systems is highly dependent on core systems that pre-exist a new system. You can learn all the things on paper about layer 8 vs layer 4 load balancing, how to build an effective distributed cache, how to building a decentralized rate limiter, etc but all of these things are highly dependent on what exists prior, and knowing how to work in non-greenfield components reflects an internal familiarity with them. The other side to systems design is more analogous to producing a map of the world; eg: documenting systems as you work on them. When I was a new engineer, this is how I learned systems design. Every new system I encountered I would begin to model. This had the added benefit of teaching me UML and how to make designs that highlight the important parts.
- uaas 4y agoWhat is layer 8?
- valbaca 4y agoThey must've meant Layer 7 / Application Layer. There is no Layer 8 in OSI model (unless they're talking about a different model)
- kodah 4y agoI did mean Layer 7. I've been calling it Layer 8 for a long time now, lmao. Not sure how that got mixed up and made permanent.
- valbaca 4y agoThere are alternative/extended models out there. Perhaps you ran into one of those?
- kodah 4y agoEntirely possible; I've worked in the intersection of Software Engineer and Systems Engineer most of my career, so that leads me to think maybe I'm prioritizing some of the stuff I learn and that older things I've learned have degraded. I say that mainly because I learned the OSI model very early on in my career.
- kristjansson 4y agoLayer 8 load balancing is humans opening and closing browser tabs, I presume?
- __d 4y agoLayer 8 is generally called the Political Layer.
- ransom1538 4y agoThe only way I learned was the "main guy" flipped off management and walked out. You start learning real fast. "WTF IS PXE BOOT. Session vms aren't booting. Oh jesus, JEFF! HA_ERR_KEY_NOT_FOUND people can't login ". I couldn't learn anything otherwise due to nerves, PR gating and risks of catastrophic failure. But, when I was the only person left, let's roll it. PR approved!
- AndyNemmity 4y agoSame thing happened to me! Entire career is based on a guy leaving, and being forced to use history commands to learn everything. lol.
- nostrademons 4y agoNote that system design interviews are often completely customized and adapt on the fly to the applicant's responses, so there's no way to do a "cracking the system design interview" the way you can with coding problems. When I give coding interviews I have one question that I give over and over again, learn the ins & outs of the questions and how candidates go wrong, and score based on whether the candidate can complete it, or how close they came to a valid solution if they couldn't. When I give system design interviews I pick a topic that's something that both the candidate & I have had experience with, start out with "How would you design X?", and see where the candidate goes, drawing on my background knowledge to prompt further about cases to consider and situations that might come up in the real-world implementation of the system. The candidate is generally driving the interview, because at the level where system design interviews become necessary, the candidate should know how to pro-actively anticipate problems.
- valbaca 4y ago> there's no way to do a "cracking the system design interview" the way you can with coding problems I'd disagree: - System Design Interview – An insider's guide: https://smile.amazon.com/gp/product/B08CMF2CQF https://smile.amazon.com/gp/product/B08CMF2CQF - System Design Interview Vol 2: https://smile.amazon.com/System-Design-Interview-Insiders-Guide/dp/1736049119 https://smile.amazon.com/System-Design-Interview-Insiders-Gu... - Designing Data-Intensive Applications: https://smile.amazon.com/gp/product/1449373321 https://smile.amazon.com/gp/product/1449373321 - System Design Primer: https://github.com/donnemartin/system-design-primer https://github.com/donnemartin/system-design-primer Especially: https://github.com/donnemartin/system-design-primer#how-to-approach-a-system-design-interview-question https://github.com/donnemartin/system-design-primer#how-to-a...
- newsclues 4y agoGiven the issues of BGP this is a great place to start an open education system!