8 ms·

I'm not very comfortable with storage engines that directly build on memory mapped files. MongoDB's current storage engine is mmap based and it's sub optimal at

by remon 12y ago

I'm not very comfortable with storage engines that directly build on memory mapped files. MongoDB's current storage engine is mmap based and it's sub optimal at best which is undoubtedly part of the reason they're building a completely new storage engine now (WiredTiger).

hyc_symas 12y ago
Using mmap well takes great care. MongoDB was careless. There's good reason to believe the MDBM designers were careless too.
- deleted 12y ago
  [deleted]
- luckydude 12y ago
  MDBM guy here. Care to elaborate on what we got wrong?
  - hyc_symas 12y ago
    See below. https://news.ycombinator.com/item?id=8734356 https://news.ycombinator.com/item?id=8734356
    - cbsmith 12y ago
      Umm... the lack of transactional integrity is part of the mdbm design. So it's only "careless" in the strictest sense of the term (the designers explicitly wanted to exploit not having to care about it). mdbm is certainly not without limitations, but is careful about its use of mmap to an extent that comparisons with MongoDB are laughable.
      - hyc_symas 12y ago
        I have already admitted my obvious biases, but seriously - when you design such a trivially corruptible system, you can't call it a persistent database; it's at most a cache. It won't survive a system crash at all, it probably won't survive an application crash intact either. To call it persistent is laughable.
        cbsmith 12y ago
        > I have already admitted my obvious biases, but seriously - when you design such a trivially corruptible system, you can't call it a persistent database; it's at most a cache. It won't survive a system crash at all, it probably won't survive an application crash intact either. To call it persistent is laughable. There are a number of use cases where in the event of a node failure it is better to rebuild from a replica or a log. Statistically, the RAM on another host is actually more reliable than local storage. Additionally, the database does have sync'ing primitives that allow for a variety of persistence strategies... just not the traditional ACID strategy. In practice, there are lots of cases where the freedom to ignore transactional integrity is very handy, and yes, a cache would definitely be one of them.
        hyc_symas 12y ago
        If DB updates are not atomically visible, then no sync'ing strategy will protect you from corruption during a crash. So in other words, you're saying "MDBM is not a persistent database." Glad that's clear. Totally agree, there are probably lots of use cases for it. But persistent data store isn't one of them.
        luckydude 12y ago
        MDBM is a hash, a pretty scalable one and fairly light weight. It is so fast that a common usage is to log updates to a log file and just rebuild the hash from that after a reboot. We've been doing that for years, works well for us. I'm guessing you are one of the people behind some other technology. Goody for you but do you really think you make your case by dissing anything else? If you have a solution that works for you, great. SGI, Yahoo, and other companies have found a use for MDBM. SGI was using it ~20 years ago and at the time there was nothing that came close to the same performance.
        nieve 12y ago
        It doesn't mean he's wrong, but hyc_symas is the CTO of Symas who produce the Lightning memory-mapped database, so part of his business is in direct competition. He does dance around this a bit below, but I think how aggressively he's trashing MDBM with broad & unsupported statements in this thread doesn't really speak well of his company or his personaly integrity. You can have a technical disagreement without looking like you're spreading FUD in place of valid technical arguments. This isn't how you do that.

by remon 12y ago

hyc_symas 12y ago

Using mmap well takes great care. MongoDB was careless. There's good reason to believe the MDBM designers were careless too.

deleted 12y ago
[deleted]
luckydude 12y ago
MDBM guy here. Care to elaborate on what we got wrong?
- hyc_symas 12y ago
  See below. https://news.ycombinator.com/item?id=8734356 https://news.ycombinator.com/item?id=8734356
  - cbsmith 12y ago
    Umm... the lack of transactional integrity is part of the mdbm design. So it's only "careless" in the strictest sense of the term (the designers explicitly wanted to exploit not having to care about it). mdbm is certainly not without limitations, but is careful about its use of mmap to an extent that comparisons with MongoDB are laughable.
    - hyc_symas 12y ago
      I have already admitted my obvious biases, but seriously - when you design such a trivially corruptible system, you can't call it a persistent database; it's at most a cache. It won't survive a system crash at all, it probably won't survive an application crash intact either. To call it persistent is laughable.
      - cbsmith 12y ago
        > I have already admitted my obvious biases, but seriously - when you design such a trivially corruptible system, you can't call it a persistent database; it's at most a cache. It won't survive a system crash at all, it probably won't survive an application crash intact either. To call it persistent is laughable. There are a number of use cases where in the event of a node failure it is better to rebuild from a replica or a log. Statistically, the RAM on another host is actually more reliable than local storage. Additionally, the database does have sync'ing primitives that allow for a variety of persistence strategies... just not the traditional ACID strategy. In practice, there are lots of cases where the freedom to ignore transactional integrity is very handy, and yes, a cache would definitely be one of them.
        hyc_symas 12y ago
        If DB updates are not atomically visible, then no sync'ing strategy will protect you from corruption during a crash. So in other words, you're saying "MDBM is not a persistent database." Glad that's clear. Totally agree, there are probably lots of use cases for it. But persistent data store isn't one of them.
        luckydude 12y ago
        MDBM is a hash, a pretty scalable one and fairly light weight. It is so fast that a common usage is to log updates to a log file and just rebuild the hash from that after a reboot. We've been doing that for years, works well for us. I'm guessing you are one of the people behind some other technology. Goody for you but do you really think you make your case by dissing anything else? If you have a solution that works for you, great. SGI, Yahoo, and other companies have found a use for MDBM. SGI was using it ~20 years ago and at the time there was nothing that came close to the same performance.
        nieve 12y ago
        It doesn't mean he's wrong, but hyc_symas is the CTO of Symas who produce the Lightning memory-mapped database, so part of his business is in direct competition. He does dance around this a bit below, but I think how aggressively he's trashing MDBM with broad & unsupported statements in this thread doesn't really speak well of his company or his personaly integrity. You can have a technical disagreement without looking like you're spreading FUD in place of valid technical arguments. This isn't how you do that.