If our current site got a 10x spike in traffic with little to no warning, the only thing we could do to prevent it from keeling over would be to raise the cache

If our current site got a 10x spike in traffic with little to no warning, the only thing we could do to prevent it from keeling over would be to raise the cache timeout on our caching proxy. Why? Well, we could easily add more application servers as needed. That would take about 15 minutes, and we'd probably only need to increase the count by about 50%, as we've got plenty of spare capacity. The real problem is our database server. Or rather, the way our CMS uses the database. Any sort of traffic increase to the CMS hammers the DB to the point of deadlocks (yes, on reads) for other portions of the site. We have some plans to improve the situation (including changing some stupid DB config decisions made a decade ago), but nothing that can be implemented short term. Thankfully, the CMS content is fairly static, and our 1-min caching isn't very aggressive. Increasing the cache timeout to 10-15 min would result in far less backend traffic and get us through most traffic spikes. The rest of our site either is available only to paying users (and thus far less likely to be significantly affected by a traffic spike) or served primarily by other data stores with much more room for capacity growth in their present configuration.