4 ms·

I know I shouldn't care but I am always weary of doing business with any company using MongoDB. 10gen put out claiming MongoDB was so much faster than SQL solu

I know I shouldn't care but I am always weary of doing business with any company using MongoDB. 10gen put out claiming MongoDB was so much faster than SQL solutions but it seemed obvious to me than turning off fsync would make the SQL solutions run at about the same speed. Plus, why would I want to run my database in mode where it is easy to lose data? MongoDB may be a useful product but their marketing is deceptive, which will lead to companies using it in inappropriate situations.

rogerbinns 15y ago
Have you noticed that the problem isn't MongoDB? Every case I've seen has been the data traffic exceeding the capacity of the system. There is no database that works well under those circumstances. Another way of putting it is that these businesses have been sufficiently successful that they have outgrown their own planning and deployment. As for your second paragraph, MongoDB has had journalling for quite a while, so you can make you writes durable and limited to the speed of your storage.
- thibaut_barrere 15y ago
  > Every case I've seen has been the data traffic exceeding the capacity of the system. There is no database that works well under those circumstances. Is there a benchmark somewhere comparing the memory/disk consumption of MongoDB vs. other datastores? If there's a significant overhead (and my early tests tend to show that there was - but I didn't make a strict benchmark though), then it would become very related to MongoDB then. (honest and real question, I'm a MongoDB user btw, as well as Redis, MySQL, Postgresql etc).
  - rogerbinns 15y ago
    The main overhead for MongoDB's storage is that the "column name" (keys) is stored in every record rather than just once as with traditional SQL database and some of the other NoSQL solutions. That is why you'll often see developers using very short key names, and one use for an "ORM" to translate between developer friendly names and short stored names. Of course this can be solved fairly easily by the MongoDB developers by having a table mapping between short tokens/numbers and the long names. This is the ticket: https://jira.mongodb.org/browse/SERVER-863 https://jira.mongodb.org/browse/SERVER-863 This is someone's measurements with different key names: http://christophermaier.name/blog/2011/05/22/MongoDB-key-names http://christophermaier.name/blog/2011/05/22/MongoDB-key-nam...
    - thibaut_barrere 15y ago
      Thanks for the links. My question goes further though, as someone who has worked with, and implemented too, column-based stores: I'm curious to compare the respective space/ram consumption for the data part, too. I think I'll write such a benchmark one day :)
- willvarfar 15y ago
  As I remember it: The journaling is only fsynced every so-often - its not like it magically gives you anything like the D in ACID. This leaves you relying on replication for durability. And everyone has problems when they lose one of the cluster. Luckily it wasn't the lot: http://blog.empathybox.com/post/19574936361/getting-real-about-distributed-system-reliability http://blog.empathybox.com/post/19574936361/getting-real-abo... I would love to be corrected; we'd all sleep easier.
  - rogerbinns 15y ago
    How about the actual facts: http://www.mongodb.org/display/DOCS/Journaling http://www.mongodb.org/display/DOCS/Journaling So yes it is periodic by default (in the millisecond range). However you can wait on any request until it has become durable. As for replication, people seem to have some hate for that, but the reality is that a journalled system that has failed (any database/operating system) will take a large amount of time to come back up, replay/recover journals etc. Not that useful.
cnagele 15y ago
Hi everyone. Chris here from Postmark. I would say that MongoDB durability used to be an issue, but now with journaling and replica sets it's not as much of a concern. There were two reasons why the secondary was less capable than the primary. First, the data had become very fragmented due to our frequent purging. And second, we were in the middle of an upgrade to our servers and that one had not been tackled yet. The primary failure came at a bad time. I could have clarified that better in the post. Regarding capped collections, yes they are faster. The problem is that they can't be sharded. With our dataset that would not allow us to scale.

rogerbinns 15y ago

Have you noticed that the problem isn't MongoDB? Every case I've seen has been the data traffic exceeding the capacity of the system. There is no database that works well under those circumstances. Another way of putting it is that these businesses have been sufficiently successful that they have outgrown their own planning and deployment. As for your second paragraph, MongoDB has had journalling for quite a while, so you can make you writes durable and limited to the speed of your storage.

thibaut_barrere 15y ago
> Every case I've seen has been the data traffic exceeding the capacity of the system. There is no database that works well under those circumstances. Is there a benchmark somewhere comparing the memory/disk consumption of MongoDB vs. other datastores? If there's a significant overhead (and my early tests tend to show that there was - but I didn't make a strict benchmark though), then it would become very related to MongoDB then. (honest and real question, I'm a MongoDB user btw, as well as Redis, MySQL, Postgresql etc).
- rogerbinns 15y ago
  The main overhead for MongoDB's storage is that the "column name" (keys) is stored in every record rather than just once as with traditional SQL database and some of the other NoSQL solutions. That is why you'll often see developers using very short key names, and one use for an "ORM" to translate between developer friendly names and short stored names. Of course this can be solved fairly easily by the MongoDB developers by having a table mapping between short tokens/numbers and the long names. This is the ticket: https://jira.mongodb.org/browse/SERVER-863 https://jira.mongodb.org/browse/SERVER-863 This is someone's measurements with different key names: http://christophermaier.name/blog/2011/05/22/MongoDB-key-names http://christophermaier.name/blog/2011/05/22/MongoDB-key-nam...
  - thibaut_barrere 15y ago
    Thanks for the links. My question goes further though, as someone who has worked with, and implemented too, column-based stores: I'm curious to compare the respective space/ram consumption for the data part, too. I think I'll write such a benchmark one day :)
willvarfar 15y ago
As I remember it: The journaling is only fsynced every so-often - its not like it magically gives you anything like the D in ACID. This leaves you relying on replication for durability. And everyone has problems when they lose one of the cluster. Luckily it wasn't the lot: http://blog.empathybox.com/post/19574936361/getting-real-about-distributed-system-reliability http://blog.empathybox.com/post/19574936361/getting-real-abo... I would love to be corrected; we'd all sleep easier.
- rogerbinns 15y ago
  How about the actual facts: http://www.mongodb.org/display/DOCS/Journaling http://www.mongodb.org/display/DOCS/Journaling So yes it is periodic by default (in the millisecond range). However you can wait on any request until it has become durable. As for replication, people seem to have some hate for that, but the reality is that a journalled system that has failed (any database/operating system) will take a large amount of time to come back up, replay/recover journals etc. Not that useful.

cnagele 15y ago

Hi everyone. Chris here from Postmark. I would say that MongoDB durability used to be an issue, but now with journaling and replica sets it's not as much of a concern. There were two reasons why the secondary was less capable than the primary. First, the data had become very fragmented due to our frequent purging. And second, we were in the middle of an upgrade to our servers and that one had not been tackled yet. The primary failure came at a bad time. I could have clarified that better in the post. Regarding capped collections, yes they are faster. The problem is that they can't be sharded. With our dataset that would not allow us to scale.