3 ms·
Since 2005, hosting has gotten much cheaper, and techniques for storing extremely large amounts of data have gotten much more efficient. But as far as I know, M
by jchw 9y ago
Since 2005, hosting has gotten much cheaper, and techniques for storing extremely large amounts of data have gotten much more efficient. But as far as I know, MediaWiki is no better positioned to begin taking advantage of those facts. It's still a PHP app accompanied by a huge, vertically scaling MySQL. There's no telling what might work better for Wikipedia, but there's plenty of avenues for experimentation. Maybe revision history could be more effectively stored in Amazon S3-like storage, instead of MySQL. Maybe not, but there's many such possible optimizations I doubt have ever seriously been considered.
- toomuchtodo 9y agoI think the savings would be in having an efficient core with lots of caching, similar to the Stackoverflow architecture. Push as much off to the CDN as possible, and dynamically reduce the cache time as edit velocity increases on a page (with cache time increasing again as edits fall off).
- zrm 9y agoYou can do even better than that with something read-mostly like Wikipedia. Have a bunch of caches (or a cache hierarchy) which request pages on first use and then cache them indefinitely. Then when a page actually changes, you push notify all the caches to immediately evict that page.
- mpartel 9y agoSounds error-prone if the notification ever fails to reach some subset of the caches. A long expiration time (e.g. several hours) is probably almost as good, but also "self-healing".
- zrm 9y agoGiving out the wrong page for several hours is obviously better than giving out the wrong page forever, but it's still broken. Better would be to re-request the page after several hours and compare it to the cached copy. Then if they're different, update the cache, but also generate an alert to notify the administrator that something "impossible" happened and there is a bug somewhere that needs to be fixed.