3 ms·
In practice there's a bit more to it than what the article describes, especially for a distributed cache: you need to have the Promise auto-expire so that if th
by thamer 8y ago
In practice there's a bit more to it than what the article describes, especially for a distributed cache: you need to have the Promise auto-expire so that if the machine that's performing the backend read disappears the other readers don't stay stuck forever waiting for it. It's also useful to have a feature in the cache itself that blocks the caller until the Promise has been fulfilled, so as to avoid repeated requests asking if the data is finally there.
As an aside, Guava loading caches[1] implement per-key locking so that multiple threads accessing a missing key simultaneously[2] would only lead to a single load with all other readers waiting for the loading thread to complete the fetch and populate the cache.
[1] https://github.com/google/guava/wiki/CachesExplained https://github.com/google/guava/wiki/CachesExplained
[2] in the sense of "in the time it takes for the first accessor to complete the backend read"
- davinic 8y agoWhy block the caller? Subsequent calls will have the same promise returned and notification will happen for all once the promise is resolved.
- deleted 8y ago[deleted]
- vhost- 8y agoBecause then your caller will need to implement something that understands what a promise is instead of just getting a data object it already has to understand. Not only this, but the caller will need to also implement a polling mechanism to keep trying. Put into code: func get_value(key): value = backend.get(key) return value Is way better than something along these lines: func get_value(key): while true: response = backend.get(key) if response.type == "promise": sleep duration continue return response.value If your service looks like the former, then suddenly your unit tests can use a postgres database, a sqlite database, a rest client... that all implement the same backend.get(key) interface.
- NovaX 8y agoGuava cache's successor, Caffeine [1], handles the asynchronous case. In your scenario, you could set a timeout on the future as the entry will be discarded if the future results in an error. [1] https://github.com/ben-manes/caffeine/wiki/Population https://github.com/ben-manes/caffeine/wiki/Population
- nitrogen 8y ago> so that multiple threads accessing a missing key simultaneously[2] would only lead to a single load with all other readers waiting for the loading thread to complete For anyone using Rails, Rails internal cache does this on expiration, but might not on initial load.