When working on most things, I ask two basic questions: how does it fail? how does it scale? I picked that up years and years ago. Any line, function, workflow

When working on most things, I ask two basic questions: how does it fail? how does it scale? I picked that up years and years ago. Any line, function, workflow, etc can fail. You need to have worked through all the failure cases and know how you are going to handle them. Somethings don't need to scale, ever. But most things I work on do. If I don't know the story on how I can ramp up an implementation a few orders of magnitude, then I can't say I've designed it well. Aside from that, measure everything. Metrics, telemetry, structured logging. Design for failure and design for understanding what happened to cause the failure. Data will tell you what you are doing, not what you think you are doing. There was a post here yesterday about a neat map thing. I hit a bug and the author could only rely on their experience with it being potentially slow at times but saying it should work. If there were proper metrics in place, they would know that N% of calls exceed M seconds and cause a timeout. He could then relate this to the underlying API and its performance as the service experiences it. With proper logging, they see what the cache hit ratio is and determine if new cache entries should be added. Build, measure, learn. Oh, and automated tests. So much to be written on that.