4 ms·
Is it possible to make Badger completely in-memory? I.e., disable persistence.
by folex 8y ago
Is it possible to make Badger completely in-memory? I.e., disable persistence.
- captn3m0 8y agoHow about using tmpfs?
- amelius 8y agoYou might as well use a simple hash-table then. It seems that most of the design effort of this project was focused on fast disk access.
- grumpydba 8y agoFast disk access on linux means async io+direct io, which badger does not use. Files are essentially mmap'd. Regarding the hash table suggestion, you would lose compression/compaction and indexing/prefix scans.
- amelius 8y ago> Files are essentially mmap'd. Yes, but using LSM tree has a performance penalty over plain hash tables. Yes, you would lose compression. But you could still have prefix scans by replacing the hash table by e.g. a red/black tree.
- mrjn 8y agoMmaping is optional. Badger doesn't require it, it gives you that option. Async IO in Go has been debated vigorously over the years, but there're little benefits in building that library. Because Goroutines do an equivalent job. Which is what we're doing, random reads (for value log access) spread over many goroutines.
- grumpydba 8y agoMy bad, I just glanced at the code. Are you using O_DIRECT? Goroutines do an equivalent job but write is synchronous and thus blocks one thread of the go scheduler, as far as I can tell. Also the benefit of the async io API is that it allows sending multiple requests in ones call. Syscalls in go do have an overhead which can be mitigated this way, I guess.
- shirakawasuna 8y agoYou'd also lose notions of transactions with a simple hash map - it can be nice to have some built-in locking / ACID-ish features, even in-memory (I use SQLite with :memory: for just that purpose sometimes).
- shazow 8y agoRelated issue discussion: https://github.com/dgraph-io/badger/issues/377 https://github.com/dgraph-io/badger/issues/377
- mrjn 8y agoYou can keep both the value log and LSM tree in memory, exposed by badger.Options. That'd still give you persistence and crash-resilience while allowing your reads to benefit from direct memory access. If persistence is a non-goal, yeah, tmpfs would work. If you use it as an in-memory DB, then I'd recommend using the LSM options, instead of Default options.