6 ms·
I have not used this package but there is another one for Go called freecache that offers a similar solution. The primary benefit of such solutions is that if y
by deckarep 7y ago
I have not used this package but there is another one for Go called freecache that offers a similar solution. The primary benefit of such solutions is that if you have a Go service where you want to rely on the local caching of data, using a package like this can significantly reduce GC scan pressure removing the work that Go needs to do when analyzing pointer data sitting on the heap.
Why does this help? Instead of keeping millions upon millions of items alive on the heap (leading to Go having to scan all such data) you can instead serialize/deserialize the data in a solution like this with usually minimal overhead. Storing your cached data in a solution like this suddenly gets rid of the need to have live pointers of data on the heap. This is because your data is now stored as []byte slice somewhere in the Cache data structure that this code uses. Finally, since packages like BigCache/Freecache are built with only a very small handful of heap objects the Go runtime performance can now go back to what it does best which is spending most of its time in your application logic.
If anyone has any doubts of this approach try it out...we saw dramatic differences with using a package like this vs a naive map of pointer based data or vs something like Hashicorps LRU datastructure.
The last service we applied this model changed CPU profile from running at around 900% to 400%. That was a big win in my book and practically cut our cluster size in half.
- gfs 7y ago> Storing your cached data in a solution like this suddenly gets rid of the need to have live pointers of data on the heap. So if I understand this correctly, Go does not do any recursive scanning of structures? Each unique bit of data owns it data for as long as it needs to?
- deckarep 7y agoNot quite, the difference here is that instead of storing live objects on the heap you would need to serialize them into a byte slice. You then hand that to the cache library and it would find a place to copy those bytes and store them on your behalf. This means your object now just becomes data belonging to the cache and sits somewhere on the libraries internal data-structure.
- erik_seaberg 7y agoPeople considering this should benchmark both options. Constantly deserializing everything you need has a predictable but very high cost, often worse for throughput than a well-tuned GC.