3 ms·
Each new value is written to the journal. Later, when the journal is compacted, the value will be written to an SSTable. Each subsequent compaction involving th
by snewman 15y ago
Each new value is written to the journal. Later, when the journal is compacted, the value will be written to an SSTable. Each subsequent compaction involving that value causes the value to be written (copied) yet again. So over time, a 100-byte value will consume k * 100 bytes of disk write bandwidth (and (k-1) * 100 bytes of read bandwidth).
In principle, k can be arbitrarily large (over a sufficiently long time span). The actual value depends on many factors. For instance, if your write rate is high in comparison with the amount of memory available for the memtable, then you will have more compactions. If data has a short half-life, then it will not survive to participate in as many compactions.
- jemfinch 15y agoIn principle, k is limited by O(log n), where n is the number of changes to the database; it's not quite accurate to call it "arbitrarily large".
- leif 15y agoCounting bytes is the wrong thing to do (for old disks). If you count disk seeks (as you should), LSM trees are a lot better than B-trees, but it turns out you sacrifice more read performance than you should. On SSDs, you do want to count bytes, and in this case, LSM trees do exhibit write amplification and it is a big deal. When your disks have a limited lifetime, even doing 3x the writes you need to hurts a lot. (Not trying to spam, I just don't want to type the same thing twice, but) I explain this better here: http://www.tokutek.com/2011/10/write-optimization-myths-comparison-clarifications-part-2/ http://www.tokutek.com/2011/10/write-optimization-myths-comp...