4 ms·
To me this is all nice and straightforward except for the merge i.e. when does it happen? Magically in the background or only at restart. In either case then t
by rozim 16y ago
To me this is all nice and straightforward except for the merge i.e. when does it happen? Magically in the background or only at restart.
In either case then the question is how does this affect the responsiveness of the system.
This paper http://downloads.basho.com/papers/bitcask-intro.pdf http://downloads.basho.com/papers/bitcask-intro.pdf is a nice easy ready with another level of detail over the linked to article.
I suspect they could consider
- varint encoding of the key and value sizes on disk
- aligning some writes to disk block boundaries to avoid spanning 2 blocks
though both probably only have minor gains.
For more inspiration see Jeff Dean's comments on the SSTable data structure at Google: http://osdi2006.blogspot.com/2006/10/paper-bigtable-distributed-storage.html http://osdi2006.blogspot.com/2006/10/paper-bigtable-distribu...
- metabrew 16y agoRiak has "windowed merges", ie you can schedule compaction at your off-peak times. Or not at all, I guess, if you never delete or modify existing data.
- siculars 16y agothe window merge is a new thing in riak 0.14, i believe. and you are correct the merge is only a consideration based on your use case. if you are using riak as a pure dump of immutable data you will not need to merge. if, however, your use case consists of a number of edits you should consider a more aggressive merge strategy. As all updates/deletes are appended to bitcask and are actually writes, your dead bytes will start growing rapidly.