I once solved worst-case write performance in an open source MVCC data store (Apache Lucy, a “loose C” port of Lucene) like so: 1) Limit the maximum size of an

I once solved worst-case write performance in an open source MVCC data store (Apache Lucy, a “loose C” port of Lucene) like so: 1) Limit the maximum size of an update. This causes gradual degradation of the index, which is ordinarily dealt with through periodic consolidation — but now this this consolidation will be disabled in the main write mechanism. 2) Enable a background consolidation process which runs concurrently. At the end of the consolidation process a short write lock will need to be obtained while you replay all deletions that have occurred since consolidation started, but the time required isn’t problematic. This fix required major surgery to the Lucy indexer core code, so it’s something that that would only be feasible if the data store itself supported it.