4 ms·

Distributed cache and K/V store in Go. Embeddable or independent service

23 points by mastabadtomm 7y ago

ddorian43 7y ago
Why not patch/fork redis and include that as a cache that can be used in different languages and has ~all redis features ?
- kbumsik 7y ago
  Is Redis embeddable? If not, it is sometimes building a whole new project is much easier and maintainable than a fork.
- mastabadtomm 7y ago
  Olric is not Redis. Redis implements different data structures and has different persistence options. Olric only provides a key/value store and it's suitable to be used as a distributed cache. README.md file contains enough information to compare those tools.
jrockway 7y ago
https://github.com/boltdb/bolt https://github.com/boltdb/bolt seems very similar.
- mastabadtomm 7y ago
  Olric is an in-memory and distributed key/value store. Bolt is an b-tree based on-disk key/value store and it's not distributed. Olric can be compared with https://github.com/golang/groupcache https://github.com/golang/groupcache and it's highly inspired by Amazon DynamoDB paper and Hazelcast.
marcinzm 7y ago
Didn't Discord have issues when they did caching in Go due to the garbage collector having to run over all objects ever 2 minutes? Was that resolved in a newer version of Go or does this project get around that problem in another way?
- kissgyorgy 7y ago
  Brad Fitzpatrick and others pointed out that in that article they were using Go 1.6, but the garbage collector got way better since then and it would be possible that just upgrading Go would have solved the issue for them.
- hrgiger 7y ago
  Seems like this has been merged: https://github.com/golang/go/issues/35112 https://github.com/golang/go/issues/35112 https://github.com/golang/proposal/blob/master/design/35112-scaling-the-page-allocator.md https://github.com/golang/proposal/blob/master/design/35112-...
  - apta 7y ago
    As far as I know, without a generational or regional GC, golang will always have issues with large heaps.
- mastabadtomm 7y ago
  I don't know the Discord case but Olric implements an append-only log file indexed with a built-in map(uint64 > uint64) to store data.
  - wlll 7y ago
    It was discussed here https://news.ycombinator.com/item?id=22238335 https://news.ycombinator.com/item?id=22238335
  - marcinzm 7y ago
    Ah, as I understood it the Discord issues only shows up when you do a large long term map(? -> object) but maps with primitive values don't trigger it.
continuations 7y ago
What data structure does it use - LSM tree or hash table or something else? Also if I'm using it as an embedded library does it do anything to help me to replicate the data?

23 points by mastabadtomm 7y ago

Why not patch/fork redis and include that as a cache that can be used in different languages and has ~all redis features ?

kbumsik 7y ago
Is Redis embeddable? If not, it is sometimes building a whole new project is much easier and maintainable than a fork.
mastabadtomm 7y ago
Olric is not Redis. Redis implements different data structures and has different persistence options. Olric only provides a key/value store and it's suitable to be used as a distributed cache. README.md file contains enough information to compare those tools.

jrockway 7y ago

https://github.com/boltdb/bolt https://github.com/boltdb/bolt seems very similar.

mastabadtomm 7y ago
Olric is an in-memory and distributed key/value store. Bolt is an b-tree based on-disk key/value store and it's not distributed. Olric can be compared with https://github.com/golang/groupcache https://github.com/golang/groupcache and it's highly inspired by Amazon DynamoDB paper and Hazelcast.

marcinzm 7y ago

Didn't Discord have issues when they did caching in Go due to the garbage collector having to run over all objects ever 2 minutes? Was that resolved in a newer version of Go or does this project get around that problem in another way?

kissgyorgy 7y ago
Brad Fitzpatrick and others pointed out that in that article they were using Go 1.6, but the garbage collector got way better since then and it would be possible that just upgrading Go would have solved the issue for them.
hrgiger 7y ago
Seems like this has been merged: https://github.com/golang/go/issues/35112 https://github.com/golang/go/issues/35112 https://github.com/golang/proposal/blob/master/design/35112-scaling-the-page-allocator.md https://github.com/golang/proposal/blob/master/design/35112-...
- apta 7y ago
  As far as I know, without a generational or regional GC, golang will always have issues with large heaps.
mastabadtomm 7y ago
I don't know the Discord case but Olric implements an append-only log file indexed with a built-in map(uint64 > uint64) to store data.
- wlll 7y ago
  It was discussed here https://news.ycombinator.com/item?id=22238335 https://news.ycombinator.com/item?id=22238335
- marcinzm 7y ago
  Ah, as I understood it the Discord issues only shows up when you do a large long term map(? -> object) but maps with primitive values don't trigger it.

continuations 7y ago

What data structure does it use - LSM tree or hash table or something else? Also if I'm using it as an embedded library does it do anything to help me to replicate the data?