5 ms·
I have written many tools that do something like this, very useful. Just gotta be careful with memory leaks, so (for a TS example) you might want to do somethi
by ralusek 4y ago
I have written many tools that do something like this, very useful.
Just gotta be careful with memory leaks, so (for a TS example) you might want to do something like this:
const promiseMap<key, Promise<any>> = new Map();
async function keyedDebounce<R>(key: string, fn: () => R) {
const existingPromise = promiseMap.get(key);
if (existingPromise) return existingPromise;
const promise = new Promise(async (resolve) => {
const result = await fn(); // ignore lack of error handling
// avoid memory leak by cleaning up
promiseMap.delete(key);
// this will call the .then or awaited promises everywhere
resolve(result);
});
promiseMap.set(key, promise);
return promise;
}
So that the promise sticks around for that key while the function is active, but then it clears it so that you're not just building up keys in a map.
- jdthedisciple 4y agoYou check for if(promiseMap.get(key)) and in the NO case you do promiseMap.delete(key)? Could you explain why that's necessary? (sorry probs a stupid question)
- aidos 4y agoI’m the no case, the promise is created, and then deleted after the function is resolved. I’m not entirely sure of the use of this pattern where your using a map and deleting the keys as you go - seems like you’d end up doing more work kinda randomly depending on how the keys were added. I’d just relive the whole map when I was done with it.
- ralusek 4y agoSo if the promise still exists, that means that there is an active call out for a promise using this key already, so we'll just return it. We know this because when the function that is executing finishes, we delete the promise from the map. A purpose of this might be, let's say you rebuild some structure very frequently, like hundreds of times per second or whatever. You can call this function for a given key as many times as you want while that async process is executing, and it will only execute it once, always returning the same promise.
- withinboredom 4y agoWon’t JS garbage collect orphaned references? Why is this necessary?
- stickupkid 4y agoIt does, but the map has a reference to it, so it will "leak" (in gc languages an unwanted reference is considered a leak). If this map got rather large, you could end up with a rather large heap and it would be un-obvious why at first.
- withinboredom 4y agoDo Promises hold a reference to the chain of functions that end in the result? If so, that seems like a bug.
- epolanski 4y agoA promise is just an object, it does not contain any reference to the chained functions.
- deckard1 4y agothere is some confusion here. OP is intentionally caching results of, presumably, expensive computations or network calls. It's a cache. There is no memory leak. OP just did not detail how cache invalidation/replacement happens. The 2nd comment adds a rather rudimentary mechanism of removing items from cache as soon as they are ready. You get the benefit of batching requests that occur before the resolve, but you get the downside of requests coming in right after the resolve hitting the expensive process again. Requests don't neatly line up at the door and stop coming in as soon as your database query returns. Typically you would use an LRU mechanism. All caches (memcached, redis, etc.) have a memory limit (whether fixed or unbounded, i.e. all RAM) and a cache replacement policy.
- andrewingram 4y agoCould you use a WeakMap for this instead?
- skrebbel 4y agoIf the key is a type that you expect to be GC'ed, yes, totally. If the key is a simple value type eg a string then it behaves the same as a regular Map or an object.
- andrewingram 4y agoAh yeah, good point. I've mostly used this overall pattern with something like Dataloader, where you throw away the entire cache on every request, so the GC problem doesn't crop up.
- blue_pants 4y agoPlease correct me if I'm wrong. But you'd better avoid creating extra promises for performance reasons const promiseMap<key, Promise<any>> = new Map(); async function keyedDebounce<R>(key: string, fn: () => R) { const existingPromise = promiseMap.get(key); if (existingPromise) return existingPromise; const promise = fn().then(result => { promiseMap.delete(key); return result; }); promiseMap.set(key, promise); return promise; }
- lolinder 4y agoThis is a great solution to the thundering herd problem, but isn't OP explicitly trying to cache the results for later? In that case, the promise map is a clever way to make sure the cachable value is fetched at most once, and you want the keys to build up in the map, so GC'ing them is counterproductive.
- ralusek 4y agoYa it obviously depends on the intended goal, but caching items in a map indefinitely better be done on a set of values that is known to be limited to a certain size over the lifetime of the server or it'll lead to a memory leak.
- kevinforrestcon 4y agoIf the result of the async function is not necessarily the same value every time you call it, you may want to recompute it