4 ms·

/rant/ You know what I think is "a big fat pile of suck"? This guy and his hack blog post. Congrats, you figured out what O(n) means. You know, this stuff is no

/rant/ You know what I think is "a big fat pile of suck"? This guy and his hack blog post. Congrats, you figured out what O(n) means. You know, this stuff is not magic. You can't magically say, hey database, give me everything I want and do it for me immediately. Oh, and make it open source, aka free. Oh, and on and on and on. This stuff is hard. Nothing comes for free. I don't even use mongodb in any production project and am really feeling for them. All these pseudo functional developers think they know how things should work and then get all indignant when stuff doesn't work the way they want it to or think it should. In this season of Thanks, let us give thanks to the real engineers who make these databases (and other software) and then give them out to everyone for free. Go send a polite email to their mailing lists and say "Dear OpenSource Developer, Thank you for all your hard work." Re. this specific problem - cache your data and stop crying about it or simply don't present counts to users.

cheald 15y ago
Hi, big fat pile of hack-posting author here. I know it's not magic. I'm a huge fan of Mongo. I blog about the issues I have with it, because the issues I don't have with it are far less interesting. The solution wasn't "AMG, MONGO SUX", it was "here's how I band-aided the problem". The impetus for this particular post was that it was somewhat surprising that counts were still doing full scans, even if the query conditions were on an indexed field. My expectation was that the index might have some information on how big a particular part of the index tree is (given two bounds), but it turns out that it doesn't, so you end up looking at each document and doing inspection per row. This isn't normally a problem, but the comparison is relatively slow, so the whole thing starts to drag down faster than you'd normally expect. When the problem is hidden below several levels of library/ORM candy coating, it's not going to be immediately obvious that this is a problem, especially if the queries against that field are otherwise performant thanks to proper use of indexes. It's non-obvious, and given that someone else found it interesting enough to post here, I'm apparently not the only person in the world to ever have this problem. There are multiple open tickets in the Mongo JIRA system, as well as multiple discussions on this topic on the mailing list. This isn't me whining that 10gen won't fix my software, it's an acknowledgment of the problem and steps towards fixing it. 10gen themselves have acknowledged via the mailing lists that counts are slower than they should be, and it's on the list of things to improve. Until then, we have to make do with workarounds.
- nnythm 15y ago
  I think that more than the content of your post, siculars is responding to the somewhat inflammatory title. As soon as I saw the title I thought, "Someone is going to be complaining about mongoDB having a slow operation, and it is going to be upvoted endlessly because mongoDB is webscale." Upon closer inspection, it became clear that this title is only inflammatory because it is on HN, and also that the title was not designed for HN: on your blog, it is simply a description of what the post is about. On HN, it becomes a honeypot. @cheald, clearly this isn't your fault, and you don't control the tenor of the discussion of mongoDB. What follows isn't a criticism of you, but more a general comment. I don't think that criticism of mongoDB should be stifled on HN (it clearly still has huge flaws--the lack of granularity when grabbing the write lock comes to mind), but it seems like it's a little hard to have an intelligent conversation in this context. It might make sense to encourage HN to discuss mongo in a way that it's not criticizing a performance detail (no database can do everything for everyone), eg where it can be compared against other databases. When criticizing a performance detail, it might make sense to move that discussion to the MongoDB Jira, where your voice can actually help influence decision making.

cheald 15y ago

Hi, big fat pile of hack-posting author here. I know it's not magic. I'm a huge fan of Mongo. I blog about the issues I have with it, because the issues I don't have with it are far less interesting. The solution wasn't "AMG, MONGO SUX", it was "here's how I band-aided the problem". The impetus for this particular post was that it was somewhat surprising that counts were still doing full scans, even if the query conditions were on an indexed field. My expectation was that the index might have some information on how big a particular part of the index tree is (given two bounds), but it turns out that it doesn't, so you end up looking at each document and doing inspection per row. This isn't normally a problem, but the comparison is relatively slow, so the whole thing starts to drag down faster than you'd normally expect. When the problem is hidden below several levels of library/ORM candy coating, it's not going to be immediately obvious that this is a problem, especially if the queries against that field are otherwise performant thanks to proper use of indexes. It's non-obvious, and given that someone else found it interesting enough to post here, I'm apparently not the only person in the world to ever have this problem. There are multiple open tickets in the Mongo JIRA system, as well as multiple discussions on this topic on the mailing list. This isn't me whining that 10gen won't fix my software, it's an acknowledgment of the problem and steps towards fixing it. 10gen themselves have acknowledged via the mailing lists that counts are slower than they should be, and it's on the list of things to improve. Until then, we have to make do with workarounds.

nnythm 15y ago
I think that more than the content of your post, siculars is responding to the somewhat inflammatory title. As soon as I saw the title I thought, "Someone is going to be complaining about mongoDB having a slow operation, and it is going to be upvoted endlessly because mongoDB is webscale." Upon closer inspection, it became clear that this title is only inflammatory because it is on HN, and also that the title was not designed for HN: on your blog, it is simply a description of what the post is about. On HN, it becomes a honeypot. @cheald, clearly this isn't your fault, and you don't control the tenor of the discussion of mongoDB. What follows isn't a criticism of you, but more a general comment. I don't think that criticism of mongoDB should be stifled on HN (it clearly still has huge flaws--the lack of granularity when grabbing the write lock comes to mind), but it seems like it's a little hard to have an intelligent conversation in this context. It might make sense to encourage HN to discuss mongo in a way that it's not criticizing a performance detail (no database can do everything for everyone), eg where it can be compared against other databases. When criticizing a performance detail, it might make sense to move that discussion to the MongoDB Jira, where your voice can actually help influence decision making.