3 ms·
> What kind of configuration change would reduce storage but overload the system? Maybe the increased storage usage was due to wrong kind of load rebalancing?
by bostik 8y ago
> What kind of configuration change would reduce storage but overload the system?
Maybe the increased storage usage was due to wrong kind of load rebalancing? I can easily see that a change to reduce _object_ hit rate in any given location, might just increase the _metadata_ hit rate. If storage is balanced by spreading the load to workers that keep their data nearby (for reduced latency), then an unexpected increase in these workers could also cause an increase in required storage.
If you want to balance the load on these properly, you'll have to do more location data lookups. Oops. Now your metadata's metadata is the chokepoint. Massive read contention on blob store address lookups --> cascading failure.