7 ms·
These days when everything can live in RAM, just focus on writing a good, durable transaction log and use an in-memory data structure. That way you won't waste
by weeksie 8y ago
These days when everything can live in RAM, just focus on writing a good, durable transaction log and use an in-memory data structure. That way you won't waste all your time re-implementing a B-tree and can focus instead on the query engine and data model. At least if you're doing it for your own gratification (and even if you're not, it may make sense to build the initial prototype that way.)
Just my 2¢
—signed, someone who went down that road a decade ago
- eismcc 8y agoAs someone who works on an in-memory DB, good advice. You can go far, but it’s worth designing your system with the notion that data could be paged in/out behind the scenes.
- setr 8y agoWhat is there to do with knowledge of paging? Afaik it should be fully transparent; can you even reliably that it occurred? I’m strictly asking, to be clear; I’m vaguely remembering an article about redis(?), describing db page-management as a poor design, as it ends up being unreliable to detect, and competes with the OS to neither’s benefit (though a similar argument is often made as to why you shouldn’t micro-optimize C — the compiler will win [except where it doesn’t])
- convolvatron 8y agoi think the key issue here is managing the asynchrony. which is sadly still a bit of a pain. you're absolutely right that you shouldn't have two page caches. but its entirely likely that the database has explicit lifetime and policy information that would make it a more effective cache owner. but sure, if you're just going to do a blind LRU, by all means leave it to the kernel.
- mavelikara 8y agoThe article you are referring to might be about Varnish.https://news.ycombinator.com/item?id=1554656 https://news.ycombinator.com/item?id=1554656
- ddorian43 8y agoOr just use lmdb.