4 ms·
Yeah, I am the engineer mentioned in the article, and I agree the explanation doesn't really work. The pieces of the explanation that ended up in the article it
by atdt 11y ago
Yeah, I am the engineer mentioned in the article, and I agree the explanation doesn't really work. The pieces of the explanation that ended up in the article itself don't add up to a coherent explanation. The fault for that is mostly mine. In hindsight, my original explanation was too long and too elaborate to be helpful. It's a good reminder that it is easy to go to far with an analogy and end up complicating the thing you were trying to simplify. Oh well, live and learn :)
There are more (coherent and to-the-point) details about PoolCounter in the prologue to PoolCounter.php in MediaWiki's source tree:
https://github.com/wikimedia/mediawiki/blob/1617e7822eaf74261687d2d2c3148ce10ab5da69/includes/poolcounter/PoolCounter.php#L3-L43 https://github.com/wikimedia/mediawiki/blob/1617e7822eaf7426...
And in a short blog post by Domas Mituzas, who is the original author of PoolCounter:
https://dom.as/2009/06/26/embarrassment/ https://dom.as/2009/06/26/embarrassment/
- homero 11y agoBasically a page view checks the cache and rebuilds if necessary. Thousands of page hits in the same second before the build is over starts thousands of parallel rebuilds.
- taf2 11y agoAlso known as the thundering herd...
- homero 11y agoYeah I never really solved it while I was using memcached but I'm not Wikipedia
- andrewingram 11y agoThe way i've seen it done is something like this: * Cache never automatically expires, but you do have some notion of staleness * Whenever you request data, you get it from the cache (if it exists), and check for staleness, so you cached data needs to know when it was cached. * Return the data as usual, but if the data was also stale, you fire off a worker to update the cached data. * If you have lots of requests happening at the same time, you have a system for seeing if a worker already exists, to ensure that you only create one (for each piece of cached data). * For the time it takes for the worker to complete, you have to be okay with serving stale data, in most cases this is okay. There's an edge case missed here, which is what to do when the cache is empty (either because it's one of the first requests, or because the cached data has been evicted). That's up to you, depending on your use case. You can basically either return a default value, you can pre-warm your cache, or you can let the requests hang until the data is ready.
- homero 11y agoI never looked into whether memcached will give you the creation date, I just let memcached expire it itself
- andrewingram 11y agoI should've fixed the formatting whilst it was still editable :( A former colleague of mine built a Django implementation of this pattern, which is pretty useful: https://github.com/codeinthehole/django-cacheback https://github.com/codeinthehole/django-cacheback
- forgotpwtomain 11y agoWhy isn't it a possibility to rebuild and invalidate the cache only when the rebuild is finished?
- homero 11y agoBecause it's usually triggered with the cache expiring then it's not available and every page view attempts rebuilds. Otherwise the cache and time will be separate. I store the cache in memcached with an expiration to detect it expiring
- forgotpwtomain 11y agoYou could queue an async job to rebuild what was stored in the cache right before the expiration time when you update / renew the expiration on the cached item?
- recivorecivo 11y agoHi. Do you think the performance of Wikipedia during this period would be improved if you had no ego? I think it would. Without an ego, a more optimal solution would result from multiple cycles of self-refutation.
- hunter2_ 11y ago+++ath0
- hedgehog 11y agoThere is a stochastic approach that can be adapted to address this problem, I think I first saw it at IMVU in 2009 but conveniently Wikipedia has a good reference now: https://en.wikipedia.org/wiki/Cache_stampede#Probabilistic_early_expiration https://en.wikipedia.org/wiki/Cache_stampede#Probabilistic_e... The advantage is less coordination is necessary and you should be able to get down to a single concurrent rerender per page.
- atdt 11y agoWow, fascinating -- thank you for the pointer!
- DanWaterworth 11y agoWow, that's a nice technique, but in this case, isn't the rendered page invalidated by changes to the source, rather than over time? I suppose, in this case you could use the time since invalidation as your input. The downside is that changes in the source aren't immediately reflected in the rendered output, especially for infrequently updated pages.