3 ms·
"One technique we repeatedly used was creating artificial extra load in the live site. For example, we would do a lot more memcached reads than necessary. Then
by maxclark 5y ago
"One technique we repeatedly used was creating artificial extra load in the live site. For example, we would do a lot more memcached reads than necessary. Then when memcached broke, we could quickly switch off the duplicate queries and have time to come up with a solution."
As much as I want to hate this, it's actually genius and elegant in its simplicity.
- altdataseller 5y agoI do something similar with disk usage. Create a dummy file with XX GB and delete it when you are running out of space
- fermuch 5y agoIIRC ext4 does this already. Saved me a couple of times.
- raffraffraff 5y agoI worked at a shop where they didn't bother with a partitioning scheme - everything was just one large volume. The amount of times I've had to do emergency cleanup on production servers...
- xxpor 5y agoAre load tests not a standard thing people do any more?
- mplewis 5y agoWhen unexpected load causes a production outage, it's often something you wouldn't have predicted using a load test.
- mapme 5y agoReminds me a bit of this AWS builders post about doing constant work as variance can lead to instability. https://aws.amazon.com/builders-library/reliability-and-constant-work/ https://aws.amazon.com/builders-library/reliability-and-cons...