4 ms·
Fast disk access on linux means async io+direct io, which badger does not use. Files are essentially mmap'd. Regarding the hash table suggestion, you would lo
by grumpydba 8y ago
Fast disk access on linux means async io+direct io, which badger does not use.
Files are essentially mmap'd.
Regarding the hash table suggestion, you would lose compression/compaction and indexing/prefix scans.
- amelius 8y ago> Files are essentially mmap'd. Yes, but using LSM tree has a performance penalty over plain hash tables. Yes, you would lose compression. But you could still have prefix scans by replacing the hash table by e.g. a red/black tree.
- mrjn 8y agoMmaping is optional. Badger doesn't require it, it gives you that option. Async IO in Go has been debated vigorously over the years, but there're little benefits in building that library. Because Goroutines do an equivalent job. Which is what we're doing, random reads (for value log access) spread over many goroutines.
- grumpydba 8y agoMy bad, I just glanced at the code. Are you using O_DIRECT? Goroutines do an equivalent job but write is synchronous and thus blocks one thread of the go scheduler, as far as I can tell. Also the benefit of the async io API is that it allows sending multiple requests in ones call. Syscalls in go do have an overhead which can be mitigated this way, I guess.
- shirakawasuna 8y agoYou'd also lose notions of transactions with a simple hash map - it can be nice to have some built-in locking / ACID-ish features, even in-memory (I use SQLite with :memory: for just that purpose sometimes).