2 ms·
The .zim file format is heavily optimized for compactness and ease of serving. For starters, it doesn't even store the original MediaWiki markup, but rather pre
by int_19h 4y ago
The .zim file format is heavily optimized for compactness and ease of serving. For starters, it doesn't even store the original MediaWiki markup, but rather pre-rendered basic HTML. Images only have the thumbnail version (the one that's shown inline when reading the article), there's no full-size to zoom in. And, of course, no edit history. Multiple articles then get bundled into clusters of ~1Mb each, and each cluster compressed using ZSTD.
https://wiki.openzim.org/wiki/ZIM_file_format https://wiki.openzim.org/wiki/ZIM_file_format
This all lets you squeeze English Wikipedia into 90 Gb. But it also makes it much more difficult to edit in-place, and, of course, no MediaWiki means that it cannot possibly work like git pull requests.
- zekrioca 4y agoI totally understand the reason why it is made for read-only consumption. However, we live in a moment where storage is significant cheaper, and so is processing. There could have a compromise, though I do not see any indication of such. SQLite could very well be used here.