3 ms·

I can wholeheartedly second natdempk's concept list, but would like to add these. They should provide more than enough inspiration. --- The failure modes; wha

by bostik 8y ago

I can wholeheartedly second natdempk's concept list, but would like to add these. They should provide more than enough inspiration. --- The failure modes; what they are and how they manifest: read contention, thundering herd, cascade, ... Mitigation strategies and where to apply them: throttling, backpressure, load-shifting, graceful degradation - and of course how to signal the upper layers that these are taking place. Decoupling reads and writes. (Search for material on CQRS and see where the rabbit hole takes you.) Error handling and tracing. This is particularly devious, because by definition error path is the unhappy path. It will be more expensive when hit, and has to spend more time serialising data. See also read contention, thundering herd and mitigation strategies. Learn and understand the differences between: telemetry, instrumentation and monitoring.

zbentley 8y ago
Those are great things to know. I wish a lot more people I worked with knew and understood them. However, I don't think they're "must know concepts for back end development". Most backend development is small in scale: a few servers, maybe more, a database, maybe something running cron/equivalent jobs or a queue worker, maybe some caches. Not much besides that. While people learning to develop systems at that scale might incidentally learn some of the above, I don't think most of those concepts will be useful until people are designing systems at much greater scale. Informally, many of them are present in small systems: e.g. when your cache server is down but connections take 10sec to time out, that's a form of backpressure, technically, but not in a super formal or useful way to understand the concept. In sum, I think those are important (essential, even) areas of knowledge for intermediate-experience back end developers, or developers looking to increase the scale of their infrastructure or projects (or work on very large-scale applications), but I don't think they're in the must-know, 101-level tier of back end knowledge. People can and do build successful, stable, easy-to-work-on back ends without knowledge of any of those things. In fact, this is the norm in our industry. The lack of knowledge of these areas is not a significant handicap (to productivity, understanding, or code/output quality) until you get beyond small scale. Most software projects do not.
- bostik 8y ago
  > I don't think they're in the must-know, 101-level tier of back end knowledge. Fair enough, I admit I am looking at things from behind glasses tinted in a certain way. But I do believe some kind of familiarity of the concepts is essential. Even at relatively small scale. Read contention in particular is really easy to hit the first time you have to deal with an increase in traffic (doesn't need to be external, could be a minor change that triples the number of cross-service calls). Simply because the first observation is that worker systems are running hot, the instinctive reaction is to add more workers. The two core problem areas in backend development - which in this context can mean anything that requires non-trivial server side processing - are I/O latency and processing capacity. Regardless of scale. Being unaware of (or worse, ignoring) them is not tenable.

by bostik 8y ago

zbentley 8y ago

Those are great things to know. I wish a lot more people I worked with knew and understood them. However, I don't think they're "must know concepts for back end development". Most backend development is small in scale: a few servers, maybe more, a database, maybe something running cron/equivalent jobs or a queue worker, maybe some caches. Not much besides that. While people learning to develop systems at that scale might incidentally learn some of the above, I don't think most of those concepts will be useful until people are designing systems at much greater scale. Informally, many of them are present in small systems: e.g. when your cache server is down but connections take 10sec to time out, that's a form of backpressure, technically, but not in a super formal or useful way to understand the concept. In sum, I think those are important (essential, even) areas of knowledge for intermediate-experience back end developers, or developers looking to increase the scale of their infrastructure or projects (or work on very large-scale applications), but I don't think they're in the must-know, 101-level tier of back end knowledge. People can and do build successful, stable, easy-to-work-on back ends without knowledge of any of those things. In fact, this is the norm in our industry. The lack of knowledge of these areas is not a significant handicap (to productivity, understanding, or code/output quality) until you get beyond small scale. Most software projects do not.

bostik 8y ago
> I don't think they're in the must-know, 101-level tier of back end knowledge. Fair enough, I admit I am looking at things from behind glasses tinted in a certain way. But I do believe some kind of familiarity of the concepts is essential. Even at relatively small scale. Read contention in particular is really easy to hit the first time you have to deal with an increase in traffic (doesn't need to be external, could be a minor change that triples the number of cross-service calls). Simply because the first observation is that worker systems are running hot, the instinctive reaction is to add more workers. The two core problem areas in backend development - which in this context can mean anything that requires non-trivial server side processing - are I/O latency and processing capacity. Regardless of scale. Being unaware of (or worse, ignoring) them is not tenable.