At around the same time I was working at a startup in London. We had two sizeable clusters - Hadoop and Cassandra - neither of which was conveniently elastic,

At around the same time I was working at a startup in London. We had two sizeable clusters - Hadoop and Cassandra - neither of which was conveniently elastic, but were located on EC2. We had sundry other instances, but those were the major culprits for our ~ £25,000 / month. It took me about a day to scope out a BOM that would well surpass that (maybe a year's expected growth) of whitebox server gear, and get some quotes from nearby co-los. IIRC hardware capex was about £15,000, and monthly rack + network something around £3,000. We relocated within a few weeks. One small bonus was predictable & consistent performance -- back then, the EU-west EC2 offerings were extremely sensitive to noisy neighbours, and our benchmarks never gave anywhere near the same results twice. I think for genuinely elastic loads, or if you're really addicted to some vendor-only services, it certainly makes sense. I suspect most customers are overly optimistic about how elastic their requirements are, and their stack can be.