5 ms·

Title is misleading, he was using the unstable dev branch. There's no evidence here that the stable branch does this.

luigi 16y ago
His excuse at the bottom of his post is way lame. It's very clear what is and isn't stable on MongoDB's main download page: http://www.mongodb.org/display/DOCS/Downloads http://www.mongodb.org/display/DOCS/Downloads
benatkin 16y ago
Flagged, but a title change would be acceptable, too.
- bravura 16y ago
  I have updated the title in response to this criticism
  - benatkin 16y ago
    Thanks! I unflagged it. Come to think of it, the article is still a good cautionary tale. What it really teaches is: a) do your research b) use the production version if you're using code in production, unless you have a really good reason not to and have some safeguards in place
piotrSikora 16y ago
The fact that it was "unstable" branch doesn't justify data loss or corruption. It can be used as an excuse to explain why server crashed, but not why data was lost. This is big no-no in my book. Also, if the 500k documents limit on 32-bit architectures is true then it's just sad.
- runT1ME 16y ago
  so one class of bugs is OK, but another is not? scratches head. Have you ever done software development?
  - piotrSikora 16y ago
    I never said that bugs are "OK", but yeah... Some bugs are acceptable and some are not. Data loss is never acceptable, especially not with databases. And yes, I did my share of software development...
    - cheald 16y ago
      Except in this case, it's not a bug, it's a design limitation. The MongoDB folks make no bones about it - the software is very limited on 32-bit systems. It warns you about it. It tells you to use 64-bit servers. If you insist on using the tool beyond the bounds for which it was designed to be used, that's your problem, as the developer. Is it a "bug" if you redline your car's engine and cause it to throw a rod? Or is that simply you pushing the machine past its design limits, despite the warnings not to do so?
      - piotrSikora 16y ago
        Data loss / corruption is design limitation?
        cheald 16y ago
        Yes. MongoDB doesn't ensure a write to disk before returning on an insert operation (it is what is called "eventually consistent") in order to maintain high write throughput. Because it uses memory-mapped files, it can't address anything larger than a 32-bit pointer on a 32-bit system, and so you would execute an insert, and when the DB eventually tries to write it to disk, it would fail due to lack of resources. Because the write is asynchronous, the client doesn't realize that the data has not been serialized to disk, and you end up with effective data loss. Fixing this would require either a) not using memory-mapped files, or b) ensuring full durability on writes, both of which would massively slow down the data store as a whole.
CitizenKane 16y ago
Not only that, but he fails to grok the reason for there being a limit with the database size on 32-bit systems. It's directly a result of MongoDB using memory mapped file IO (http://en.wikipedia.org/wiki/Memory-mapped_file http://en.wikipedia.org/wiki/Memory-mapped_file) which gets better performance, but on a 32-bit system you're constrained to the amount of memory addressable by a 32-bit process. So rather than do somewhat hacky (and complex) code they opted to be forward looking and just leave the limitation in place. Thankfully, with a 64-bit OS the upper limit goes up to 256 terabytes.

luigi 16y ago

His excuse at the bottom of his post is way lame. It's very clear what is and isn't stable on MongoDB's main download page: http://www.mongodb.org/display/DOCS/Downloads http://www.mongodb.org/display/DOCS/Downloads

benatkin 16y ago

Flagged, but a title change would be acceptable, too.

bravura 16y ago
I have updated the title in response to this criticism
- benatkin 16y ago
  Thanks! I unflagged it. Come to think of it, the article is still a good cautionary tale. What it really teaches is: a) do your research b) use the production version if you're using code in production, unless you have a really good reason not to and have some safeguards in place

piotrSikora 16y ago

The fact that it was "unstable" branch doesn't justify data loss or corruption. It can be used as an excuse to explain why server crashed, but not why data was lost. This is big no-no in my book. Also, if the 500k documents limit on 32-bit architectures is true then it's just sad.

runT1ME 16y ago
so one class of bugs is OK, but another is not? scratches head. Have you ever done software development?
- piotrSikora 16y ago
  I never said that bugs are "OK", but yeah... Some bugs are acceptable and some are not. Data loss is never acceptable, especially not with databases. And yes, I did my share of software development...
  - cheald 16y ago
    Except in this case, it's not a bug, it's a design limitation. The MongoDB folks make no bones about it - the software is very limited on 32-bit systems. It warns you about it. It tells you to use 64-bit servers. If you insist on using the tool beyond the bounds for which it was designed to be used, that's your problem, as the developer. Is it a "bug" if you redline your car's engine and cause it to throw a rod? Or is that simply you pushing the machine past its design limits, despite the warnings not to do so?
    - piotrSikora 16y ago
      Data loss / corruption is design limitation?
      - cheald 16y ago
        Yes. MongoDB doesn't ensure a write to disk before returning on an insert operation (it is what is called "eventually consistent") in order to maintain high write throughput. Because it uses memory-mapped files, it can't address anything larger than a 32-bit pointer on a 32-bit system, and so you would execute an insert, and when the DB eventually tries to write it to disk, it would fail due to lack of resources. Because the write is asynchronous, the client doesn't realize that the data has not been serialized to disk, and you end up with effective data loss. Fixing this would require either a) not using memory-mapped files, or b) ensuring full durability on writes, both of which would massively slow down the data store as a whole.

CitizenKane 16y ago

Not only that, but he fails to grok the reason for there being a limit with the database size on 32-bit systems. It's directly a result of MongoDB using memory mapped file IO (http://en.wikipedia.org/wiki/Memory-mapped_file http://en.wikipedia.org/wiki/Memory-mapped_file) which gets better performance, but on a 32-bit system you're constrained to the amount of memory addressable by a 32-bit process. So rather than do somewhat hacky (and complex) code they opted to be forward looking and just leave the limitation in place. Thankfully, with a 64-bit OS the upper limit goes up to 256 terabytes.