4 ms·

Does Wikipedia really store every single revision of every single file? As in, not deltas? Why is it done that way?

• _delirium 17y ago
  In the dump, I think for robustness and ease of extracting subsets. Robustness: Having to essentially play back a log to recover any particular revision increases a chance of something eventually getting corrupted, and so it's somewhat safer to avoid it in something intended to be archival. Ease of extracting subsets: For researchers, having the revisions be independent allows you to filter the XML dump through a SAX parser (or similar) to grab only revisions meeting particular criteria. If deltas were stored, you'd have to reconstruct those revisions from the deltas, which would make it really expensive to do things like, "I want to look at every article as it appeared at noon on April 1, 2007". In the live DB, I think just because it's cheaper to get a ton of storage, esp. for rarely-retrieved old revisions, than to add the overhead of computing deltas and applying them to reconstruct revisions. In particular, you'd have to compute a diff for every edit in that situation, whereas currently MediaWiki only computes diffs when a user requests to view one from the "history" tab, which is a tiny proportion of all edits.
  • JoachimSchipper 17y ago
    I understand it's simpler to store everything, and simplicity _is_ a virtue; but one could store the current revision plus deltas (and perhaps a few intermediate revisions for oft-edited articles), and obtain performance at least as good as in the current case. It would also save lots of space.