3 ms·

Here's what I do not understand: Every last tool I have ever worked with has trade-offs. I don't have any problem with that; I've sometimes even gone as far a

by gecko 14y ago

Here's what I do not understand: Every last tool I have ever worked with has trade-offs. I don't have any problem with that; I've sometimes even gone as far as phrasing it as, "If you don't hate your tech stack, you aren't really using it." I can tell you all kinds of things that C#, the .NET CLR, IIS, Apache, Mercurial, Git, elasticsearch, Redis, Gunicorn, Python, Celery, and SQL Server do that make me livid, because I've used them heavily. But that's because I use them heavily. I've never had any of these tools bite me in the butt early in the process, and definitely not had any of them bite me in the butt in unexpected ways down the road. They bite me when I push them incredibly hard, right to their limits, and, due to well-understood design constraints that I'm frequently anticipating hitting ahead of time, they fall down. That's normal and fine, and handling those situations is just good software engineering. Your tools will have limits, and that stinks, but handling those limits is part of what your job entails, and you need to deal with it. I do not use MongoDB. But here's what I see: about once a month, I come across an article where something incredibly fundamental to Mongo does not work properly. Not only does it not work properly: the way it doesn't work properly is exceedingly bad. In this case, Newsblur can't shard, which removes one of Mongo's best benefits, and the way it fails isn't to tell you early on that you will not go to space today, but rather to segfault and die after six hours of replication. That's not predictable. That's not documented. And that's not something you can anticipate. As a developer, that concerns me, and it should concern you, too. I understand that 10gen is an awesome, responsive company, and they have always been there to help. I don't want to malign that. When the Trello team had Mongo-related issues the other week, they were trying to help them out, too. But I genuinely do not view as paranoia my belief that the frequency and severity of stories like this mean that MongoDB is still not a good technology choice.

jrussbowman 14y ago
not to argue mongodb being good or not, but how many of these issues would be caught if the sites built and used proper dev and test environments? seems like as the cloud has caught on I have read about horrible things happening with all kinds of software that may not have happened with proper procedures. not to say this is that bad or not normal. I think it's a growing curve we are going to see as more people get involved with running and maintaining major systems.
taligent 14y ago
Remind me again why that you think MongoDB is "still not a good technology choice" ? Given that you're (a) not using it, (b) making judgements based on a series of blog posts, (c) extrapolating bugs that only manifest under specific scenarios, (d) are clearly new to technologies like Apache, ElasticSearch, Git, Redis etc so haven't seen some equally disturbing bugs.
lucian1900 14y ago
It's a really terrible database. No one should be using it for that. There are two things it's good at: 1) indexing arbitrary json and 2) searching for geo points offline.
nasalgoat 14y ago
I have to agree - if I had the choice to make again, I wouldn't have chosen MongoDB. It is a fundamentally broken product by design. For example, for their sharding setup they require three separate servers to host shard config data. The idea is to have high availability of that data should one of those config servers go away. However, by design, losing a single config server can cause multiple shard masters to die, requiring a manual restart. How they die and when is random, determined by where in a block migration a server is. How is this high availability? All three config servers must be running at all times without interruption for your cluster to be stable. Their roadmap to deal with this issue is sometime next year. One month not too long ago, my company found 90% of the bugs listed in their bug tracker for a specific release of MongoDB - many of which would have been found with a basic unit testing suite and some minor load testing. We were effectively performing QA functions for 10gen in our production environment. I've gone through nine releases of their PHP driver to deal with broken Data Center Awareness and none of them have worked - DCA still eludes them two years later. We have to do an OS level hack to make this work, that breaks other HA functions. Finally, their mmap design means that memory use is extremely inefficient - on a box with 256GB of memory, with a database that is only 100GB in size, it still hits disk on db reads because they offload memory management to the OS. Any other enterprise-level DB would preload the entire dataset in memory if there's room, but not MongoDB. It really is terrible.
- Qerub 14y ago
  > it still hits disk on db reads because they offload memory management to the OS Or rather: It hits the OS's disk cache. I'm not saying that this isn't a problem, but it's far from as bad you make it sound. (http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Serve... actually recommends limiting `shared_buffers` (PostgreSQL's in-memory cache) to let the OS disk cache do its magic.)a

by gecko 14y ago

jrussbowman 14y ago

not to argue mongodb being good or not, but how many of these issues would be caught if the sites built and used proper dev and test environments? seems like as the cloud has caught on I have read about horrible things happening with all kinds of software that may not have happened with proper procedures. not to say this is that bad or not normal. I think it's a growing curve we are going to see as more people get involved with running and maintaining major systems.

taligent 14y ago

Remind me again why that you think MongoDB is "still not a good technology choice" ? Given that you're (a) not using it, (b) making judgements based on a series of blog posts, (c) extrapolating bugs that only manifest under specific scenarios, (d) are clearly new to technologies like Apache, ElasticSearch, Git, Redis etc so haven't seen some equally disturbing bugs.

lucian1900 14y ago

It's a really terrible database. No one should be using it for that. There are two things it's good at: 1) indexing arbitrary json and 2) searching for geo points offline.

nasalgoat 14y ago

I have to agree - if I had the choice to make again, I wouldn't have chosen MongoDB. It is a fundamentally broken product by design. For example, for their sharding setup they require three separate servers to host shard config data. The idea is to have high availability of that data should one of those config servers go away. However, by design, losing a single config server can cause multiple shard masters to die, requiring a manual restart. How they die and when is random, determined by where in a block migration a server is. How is this high availability? All three config servers must be running at all times without interruption for your cluster to be stable. Their roadmap to deal with this issue is sometime next year. One month not too long ago, my company found 90% of the bugs listed in their bug tracker for a specific release of MongoDB - many of which would have been found with a basic unit testing suite and some minor load testing. We were effectively performing QA functions for 10gen in our production environment. I've gone through nine releases of their PHP driver to deal with broken Data Center Awareness and none of them have worked - DCA still eludes them two years later. We have to do an OS level hack to make this work, that breaks other HA functions. Finally, their mmap design means that memory use is extremely inefficient - on a box with 256GB of memory, with a database that is only 100GB in size, it still hits disk on db reads because they offload memory management to the OS. Any other enterprise-level DB would preload the entire dataset in memory if there's room, but not MongoDB. It really is terrible.

Qerub 14y ago
> it still hits disk on db reads because they offload memory management to the OS Or rather: It hits the OS's disk cache. I'm not saying that this isn't a problem, but it's far from as bad you make it sound. (http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Serve... actually recommends limiting `shared_buffers` (PostgreSQL's in-memory cache) to let the OS disk cache do its magic.)a