4 ms·
(co-author here) Thanks for sharing this post! We've elaborated the various issues we encountered trying to use a concurrent LRU cache in Dgraph and our dissati
by mrjn 8y ago
(co-author here) Thanks for sharing this post! We've elaborated the various issues we encountered trying to use a concurrent LRU cache in Dgraph and our dissatisfaction at existing choices which fail to provide memory management, high concurrency and hit ratios.
With guidance from Ben Manes, we're planning to work on a new concurrent Go cache based on Caffeine (Java). If you're interested in helping (or already have something similar), do reach out to us!
Meanwhile, check out Dgraph, our distributed graph database which is what keeps us busy: https://github.com/dgraph-io/dgraph https://github.com/dgraph-io/dgraph
- kochthesecond 8y agoVery cool, I have missed Caffeine when making stuff in go!
- kasey_junk 8y agoWhat are you thinking about for the API signatures? Without beating a dead horse this is where go’s lack of generics really becomes burdensome. If it were an on disk cache []byte would likely make sense but given it’s in memory I’m not sure. If you use interface{} you’ll need to measure that cost as well.
- amanmangal 8y agoOur current plan is to store []byte given that badger stores blobs of bytes. Using interface{} may have some overhead, but we are no so worried about that right now.
- kasey_junk 8y agoSo the []byte solution requires running through a serielization step which for most in memory uses will be expensive. Did you choose that because in you use case it eventually ends up setialized in any case?
- amanmangal 8y agocorrect!
- np_tedious 8y agoI'm a go novice, and don't have any particular knowledge about dgraph, but is this a case where auto-generated code could be a good solution? (Assuming it's a bounded set of types)
- Thaxll 8y agoI don't see why it's a problem, caching solutions like Redis also store bytes. If you cache a JSON blob you don't have much choice.
- kasey_junk 8y agoFor an in memory cache (ie one not going to disk/network) you have to unnecessarily serialize your in memory representation to bytes. This is very frequently the most expensive (computational) step in a system. If you are just going to send that stuff to nic/disk eventually anyway its perfectly fine (which is why something like Redis which is an out of memory cache can get away with it).
- mrjn 8y agoAs mentioned down in this thread, the key-val would be bytes-bytes or string-bytes. While this is more expensive in terms of CPU, it has the advantage that the size of elements in the cache are carefully understood and do not need the cost of GC sweeps. When de-serialized, in-memory structs take up more memory on heap and it gets hard to compute that -- we've tried that in vain in the past. Having said that, there's still a general need to store pointers as values and that's something we'll look at once this byte-based implementation is ready.
- AboutTheWhisles 8y agoYou might want to take a look at https://github.com/LiveAsynchronousVisualizedArchitecture/simdb https://github.com/LiveAsynchronousVisualizedArchitecture/si... It is a completely lock free key value store, though does not have eviction built in. Its memory management is done using a free list of blocks, which is part of how it maintains lock free concurrency.