Not the parent, but the linked material is more foundational than the subject matter raised in the post. There is in fact an appreciable lack of good, battle-te

Not the parent, but the linked material is more foundational than the subject matter raised in the post. There is in fact an appreciable lack of good, battle-tested, non-secret, sometimes-but-not-necessarily anecdotal public info about the part of the design process where you have a working system doing fairly okay, but you know you're inches away from a very unpleasant wall. On fire [1]. It doesn't help that distributed systems are a dark art, that many open source and free-to-use tools that developers have access to gate the HA/clustering features behind steep pricing (though I sympathize it's one of the few effective ways to make money in open source), and that expertise with scaling is very often a competitive advantage. [1] http://www.slideshare.net/iammutex/scaling-instagram/25-404ing_on_Djangocausing_tons_of http://www.slideshare.net/iammutex/scaling-instagram/25-404i...