3 ms·

I am using BoltDB in production. BoltDB is wonderful for mostly read operations, if you perform a lot of read/write operations, Redis will at the end be better

by Loic 11y ago

I am using BoltDB in production. BoltDB is wonderful for mostly read operations, if you perform a lot of read/write operations, Redis will at the end be better because of the B-tree of BoltDB and the way the pages are handled (the author and the docs do not hide this, heavy read is the use case). What I take from his post is that his software was nicely designed to be able to easily change the backend which will allow him to easily switch back to Redis in the future when he will get corruption from BoltDB because of the intensive read/write which is not the use case.

deforciant 11y ago
In case of Hoverfly, it never needs read/write operations, it only writes during "capture" mode and then only reads during "virtualize" mode and that's when performance really counts, so it looks like a perfect solution.
ddorian43 11y ago
BoltDB is based on LMDB which is supposed to never be corrupted, so if they based it right they'll be fine.
- justin66 11y ago
  They share no code at all.
  - benbjohnson 11y ago
    The fundamental design of Bolt is the same as LMDB though. The premise behind why LMDB is "uncorruptable" is that all the dirty data pages for a transaction are written out first followed by a write to a double-buffered meta page that points to the new root of the B+tree. If any data page is partially written then it doesn't matter because the new meta page hasn't been written to point to it. If the new meta page is partially written then it is detected through a checksum and the previous meta page is used (thereby rolling back the transaction). That's how Bolt works as well. That being said, all code has bugs (yes, even LMDB). Bolt has a large amount of test coverage as well as randomized black box testing that it uses to help minimize those bugs.
    - justin66 11y ago
      That's a really good explanation of the db's function, thanks. Are you guys still single process? That killed a few pretty important lmdb use cases. I've always been curious why you did that.
    - Loic 11y ago
      The corruption I was talking about is this one: https://github.com/boltdb/bolt/issues/348 https://github.com/boltdb/bolt/issues/348
    - hyc_symas 11y ago
      Just for the record, LMDB uses both random testing as well as explicitly targeted test cases. (I.e., we construct a DB with specific data in specific sequences to trigger splits and rebalances, etc.) Test coverage is over 90%, with the remainder being auxiliary and platform-specific code. (I.e., we cannot get 100% coverage on any single box due to segments of the code that are #ifdef'd for some other platform.)

by Loic 11y ago

deforciant 11y ago

In case of Hoverfly, it never needs read/write operations, it only writes during "capture" mode and then only reads during "virtualize" mode and that's when performance really counts, so it looks like a perfect solution.

ddorian43 11y ago

BoltDB is based on LMDB which is supposed to never be corrupted, so if they based it right they'll be fine.

justin66 11y ago
They share no code at all.
- benbjohnson 11y ago
  The fundamental design of Bolt is the same as LMDB though. The premise behind why LMDB is "uncorruptable" is that all the dirty data pages for a transaction are written out first followed by a write to a double-buffered meta page that points to the new root of the B+tree. If any data page is partially written then it doesn't matter because the new meta page hasn't been written to point to it. If the new meta page is partially written then it is detected through a checksum and the previous meta page is used (thereby rolling back the transaction). That's how Bolt works as well. That being said, all code has bugs (yes, even LMDB). Bolt has a large amount of test coverage as well as randomized black box testing that it uses to help minimize those bugs.
  - justin66 11y ago
    That's a really good explanation of the db's function, thanks. Are you guys still single process? That killed a few pretty important lmdb use cases. I've always been curious why you did that.
  - Loic 11y ago
    The corruption I was talking about is this one: https://github.com/boltdb/bolt/issues/348 https://github.com/boltdb/bolt/issues/348
  - hyc_symas 11y ago
    Just for the record, LMDB uses both random testing as well as explicitly targeted test cases. (I.e., we construct a DB with specific data in specific sequences to trigger splits and rebalances, etc.) Test coverage is over 90%, with the remainder being auxiliary and platform-specific code. (I.e., we cannot get 100% coverage on any single box due to segments of the code that are #ifdef'd for some other platform.)