5 ms·

A Depressive Journey With MongoDB

105 points by TheSmoke 15y ago

kennu 15y ago
So he did zero load testing before publishing an app live on television, developed with technology completely new to him?
- Jgrubb 15y ago
  Developed in four days, don't forget that part.
  - kennu 15y ago
    It just sounds like a huge gamble. It would be quite lucky to get everything working by chance on the first try, without any bottlenecks in any part of the newly created config.
    - GFischer 15y ago
      I guess he took away some good lessons from this, since his "morals of the story" say: * Do not accept responsibility for anything that you had to do in a very limited time. * Do not accept the job if the timeline is short, the work is big and the load is heavy. * Load test your application no matter what the cost. If they want to get them all, they need to pay for them all. Failing is hard, but I'm sure he learned :)
      - gbog 15y ago
        I'd add "don't ever change your techno 4 days before you go on TV"
- skormos 15y ago
  Seriously, the article should've been titled "Why You Can't Deploy Publicly Available Apps in Four Days".
- sylvinus 15y ago
  yes to me this is the craziest part.
samarudge 15y ago
Interesting, I recently worked on Project4Awesome, a community event on Youtube. Our site used a single MongoDB node on a relatively standard server and handled >100k pageviews in 24 hours with no caching. That being said, I've worked with Mongo before and know many of it's issues, so I knew what to expect. Though, without meaning to criticize the author too much, increasing various ulimits should be standard on pretty much any database server, I would have thought?
- po 15y ago
  It is hard to tell but it seems like he's talking about serving 1/3 of your daily traffic in a few minutes. When web servers fail, they tend to fall off a cliff.
  - lrobb 15y ago
    Yeah, I read it as 30k requests in a matter of seconds.
  - samarudge 15y ago
    Indeed, but Mongo is actually quite good at failing predictably. I think the biggest issue is not Mongo, but failing to test the application correctly. Particularly when working with a new technology. Mongo isn't the best fit for everything but nearly all the issues he describes are more down to not having enough time to plan and test before the project, and one would have probably encountered similar issues if using nearly any software at that scale, with no prior knowledge or tuning.
- kennu 15y ago
  Do you have any advice on the MongoDB issues to be aware of in this kind of usage? I wonder if there is a checklist somewhere.
  - DanielShir 15y ago
    I'd also appreciate the same checklist samarudge :)
  - 15y ago
emmapersky 15y ago
I thought MongoDB was webscale. hmmm. ;)
- dasil003 15y ago
  Yes but apparently not televisionscale.
EwanToo 15y ago
Given that his disclaimer reads: DISCLAIMER You are about to read a long story on how I got burnt with MongoDB and depressed with it. I am not blaming MongoDB, anyone using, advocating or developing it. I am blaming myself for this. MongoDB is a good tool. You can use it but just make sure it is what you need and it handles your requirements very well. This is not specific to MongoDB but applies to every tool we use. I think it's a bit harsh for people to criticise either him or MongoDB too much. At the end of the day, it was a big rush job, no time was allowed for testing for scaling, and it fell over. My only real surprise is that it wasn't Apache that fell over first.
- davidw 15y ago
  > My only real surprise is that it wasn't Apache that fell over first. The Apache web server is a very well-tested, tried and true piece of infrastructure, even if it's not "hip" these days.
  - dasil003 15y ago
    True, but it often needs tuning to handle massive loads.
  - EwanToo 15y ago
    True, but it's defaults are not really "battle ready" and since they didn't tweak MongoDB in advance, I can only guess that Apache was already pre-configured for that load - perhaps they used it for their existing web servers?
    - spydum 15y ago
      I would guess Apache httpd is a tool they are familiar with and comfortable managing. Trying to building a high performance MongoDB system in just a couple days was probably a bad first project with the tool. They really should have just stuck to what they know, until familiar with the new shiny.
- TheSmoke 15y ago
  apache had not fallen first because netscaler is load balancing between 26 web servers and they are already pre-configured for all these heavy load.
psyren 15y ago
If you have 4 days to implement something important, the last thing you should do is pick a technology you have no experience with.
meanguy 15y ago
"the only thing I had in my mind was MongoDB." Well, when you approach it like that...
jeswin 15y ago
Among reasons, I see stuff like this: "Appearantly someone set the connection timeout to 9000 secs which means a connection to a MongoDB instance through Netscaler is open for 2.5 hours. I have set it to 20 secs. I am not looking for someone to blame but probably it's our sys admin. Wouldn't any database give you plain bad performance if settings were badly configured? I understand it was merely one of the reasons, but sometimes all you need is one.
- TheSmoke 15y ago
  this was a reason specific to our stupidity. why on earth would someone keep connections open for 2.5 hours? how he misses this setting? my point in this is, if you have something that balance connections between servers, check if it kills them or not.
  - hieronymusN 15y ago
    Were you doing any connection pooling with Mongo? I'm curious what driver you were using, etc.
    - alexchamberlain 15y ago
      He gives the impression in the article that it was PHP.
  - mnutt 15y ago
    Perhaps it was just some mismatch between the expectation (pooled, long-running connections) and reality. (many, many short-lived connections) I think another takeaway from this is any time you need to adjust the ulimit, take a step back and ask "Does this make sense? Is my use case exceptional enough that it won't perform within the default limits set by my OS?"
chris123 15y ago
Sounds like less of a Mongo problem and more of a "trying to do too much, too fast, too big" problem. The Greek call this "Hubris." Great lesson/reminder to share. Thanks.
wseymour 15y ago
Stop blogging and start growing a pair. Aside from the monumental error of judgement in developing on an unfamiliar database in 4 days with zero testing, the 'morals of the story' crack me up: "Do not accept responsibility for anything that you had to do in a very limited time." Yeah, because the ability to stand by your decision is a function of deadlines. "Do not accept the job if the timeline is short, the work is big and the load is heavy." Wow, that's some pretty serious ambition right there. I also like the way these points are first on the list, before actual lessons like: "Do not have trust in any of your tools until they prove themselves."
- GFischer 15y ago
  I think that learning to stand up to people making unreasonable requests is an important lesson. I hope he also learned from his error in judgement. And he actually did have some courage in submitting his mistakes in public for us to discuss :)
  - wseymour 15y ago
    +1. I should change that opening to "stop blogging and continue growing a pair."
  - TheSmoke 15y ago
    i have written all these because i have made these mistakes so someone in a similar situation does not. it's not about courage, it's knowing yourself. when you do stupid things, you have to acknowledge so you don't do them the very next time. since the show has 5 more weeks and we will have more load, more requests, more everything, i would like to hear everyone's opinion on that.
- dextorious 15y ago
  """ "Do not accept responsibility for anything that you had to do in a very limited time." Yeah, because the ability to stand by your decision is a function of deadlines. """ It most certainly is. One can be forced to come up with something under a tight deadline that he wouldn't under normal circumstances.
level09 15y ago
This post was very helpful indeed. we are facing almost simliar issues with our default box configurations (ulimit and connection counts etc) while we are serving real time updates to our visitors.. Thanks for posting this.
rtperson 15y ago
> We could not test the application because we were told about it on tuesday and it had to be ready on sunday. Well there's your problem right there. That is not an appropriate timeframe for a new app. The only professional response to such a request is: "It cannot be done. Not if you want this app to work in production. Here's why." Follow up with an explanation of why QA and load testing are crucial. If they press you, make your next follow-up a letter of resignation.
- haasted 15y ago
  Also, if you accept the timeframe, go with only known technology. This is not a scenario for getting experience in a new platforms.
rjurney 15y ago
Bigger machines. More RAM. When in doubt, and in a rush, throw hardware at the problem.
jm4 15y ago
I don't think the author is exactly unnecessarily bashing MongoDB, but MongoDB is not the problem here. It is only part of the story because that's what the author chose to use. I would think that under these circumstances and time constraints, any solution the author might have come up with would have failed. The bottom line is there were only 4 days to go from idea to supporting 30k concurrent users. That is a recipe for disaster. There are so many things to consider and implement in addition to the software. Nevermind the fact that there were probably laughable requirements for this project and no time allocated to design and testing. There wasn't even sufficient hardware to support the application. This project was a failure almost from the moment it was conceived. This guy never even had a chance. There is a story in here, but it is not so much about MongoDB.
- TheSmoke 15y ago
  as i have said in the disclaimer, i'm definitely not bashing or blaming mongodb for anything. when you read it, you will see that i have done a 30K/sec load testing. the url it was pointed at was doing writes to mongodb. it opened every connection, wrote to the db and closed the connection. this is a huge success for mongodb. the problem is, when you send every single read query to mongodb under a heavy load, within a few seconds, it will just accept connections and will not respond to queries. this is definitely not the case for writing. writing handles them very well. i really would like to know how we screwed there.
  - jm4 15y ago
    I don't know enough about MongoDB to answer your question, and it sounds like you didn't know enough about it to even consider putting it into production for this project. That's where you got screwed. Trying out new technology when you have a likely impossible deadline and then having it fall over in miserable fashion for everyone to see is a good way to get fired. At that point, the debate is not over the ridiculous constraints placed upon you, but rather over the decisions you made. http://www.mongodb.org/display/DOCS/Connections http://www.mongodb.org/display/DOCS/Connections That page says each connection spawns a thread. That could be a good lead. The documentation suggests using a connection pool (although many drivers will do this automatically). Were you using one? http://stackoverflow.com/questions/8439639/mongodb-max-connections http://stackoverflow.com/questions/8439639/mongodb-max-conne... That StackOverflow question is about supporting 20k concurrent users. It could also be a matter of insufficient hardware. I don't know, but I would start looking at what kind of performance can be expected in different setups and go from there. The most important thing is to properly test your setup. Prepare for a process of trial and error and expect it to take some time.
benmmurphy 15y ago
how many db servers can handle 20k connections. dbs usually aren't written to handle that. most mysql implementations are going to have problems > 500.
tmountain 15y ago
I work at a company where we use Mongo to store and retrieve session information for over 2.5 million users per day. Our initial deployment crashed and burned in a similar fashion for one very simple reason. Mongo opens a thread per connection. Under this model, you simply cannot point every Apache request to Mongo and expect it to handle the load. The solution is fairly simple. You can either use persistent connections at the driver level (the PHP driver provides this, I'm not sure which other languages do), or you can put mongos (used for sharding) in front of mongod. Mongos does connection pooling and tremendously reduces the load on your mongod server. The author of the post mentions that redis is "heaven sent". Well, redis handles its connections in an asynchronous fashion which is why it can handle a large amount of direct traffic gracefully. I believe there are plans to change mongo's connection handling to use select/epoll, but I'm not sure when.
hello_moto 15y ago
I probably don't like the tone of my comment on this one but TheSmoke, please don't do this repeatedly. ever. again. While I respect your honesty, but after reading this particular blog post and a few bits of your website, I think you should slow down and value tried and tested tools. Let's start with your "About me:" "Mengu, is a web developer with Ruby on Rails, Grails, Django, Pyramid, web2py, CodeIgniter and jQuery in his tool box. Available for any kind of web development jobs. Read more.." Lots of technology there. Then you take on a project where you chose MongoDB. I think I'm starting to notice a trend here. You burned your team member for not standing up for your choice. "Our manager accused me and MongoDB for the failure while we all have agreed to use MongoDB in the team. None of my teammates stood up." My prejudice sense something is off: I felt that you were the one who pushed MongoDB hard to the point where your team member gave up and just "alright, let's just get this ball rolling since we have 4 days anyway..." rather than argue with you. It fails. You got blamed alone. None of your team member stood up for your "ignorance" of tried-and-tested tools. That's fitting if I may say so. Don't do that in public blog. Don't ever ever do that. Especially when it's your fault. You also started off with saying that "MongoDB is okay" then trash the community and not willing to purchase for the support while admitting that you do need the support. I know it has been hard for you but you really need to step back a little bit and do a retrospective on your decision-making ability. I know I'm saying harsh thinsg but I hope you understand that there's a bigger problem and MongoDB is not that problem (at least not now). In my short time in this hi-tech world, I've met and worked with a few people who religiously pushing new hip tools with zero experience (using the said tools) lately (especially lately, due to the explosion of blogs and hip tools) and would want to spread the blame to everybody else if it fails just because they agreed to use it. It's a common trend that most managers can see quickly and it may not be good for your future career.
- TheSmoke 15y ago
  hi. i'm not offended by what you said. in fact, i'm glad that you took your time and wrote to me. if i was reading a post like mine, i would also come to a conclusion like yours. unfortunately, i have to say that you are wrong. our team consists of 3 developers. there is not a single minute that i have pushed mongodb or anything else hard. this is a web site that has thousands of reqs / sec. nobody in our team can push new tools let alone pushing religiously. this is not a toy for new things to try. "i" was accused because it was me who stood up for the decision, among the team. plus, i am brave enough to take the blame when i am to blame. thanks again.
jerf 15y ago
Put this in the URL bar for the page: javascript:void(document.getElementsByClassName("post-body")[0].style.whiteSpace = 'pre-wrap')
whyme 15y ago
> THE MORAL OF THIS STORY The OP missed: When given a project that has high visibility and a very short timeline, don't wing it and choose a technology that 1. you have no experience with and 2. you've done no testing on. Sorry, but I feel the OP is failing to see he made the critical error.
mrinterweb 15y ago
I appreciate that the author put the disclaimer at the top that MongoDB is not to blame, but I do feel that for someone skimming headlines, the impression they may arrive at is MongoDB = bad.
PaulHoule 15y ago
so far I haven't been impressed with mongodb. i've seen it break down and die doing tasks that mysql does without breaking a sweat.
cagenut 15y ago
This is straight up web operations 101, has nothing to do with MongoDB. Hearing you pushing for a new tool at the last minute, then changing its setup/config/architecture repeatedly over a few days, then blaming your sysadmin for a setting. Man I'd hate to have that guys job, you just steamrolled him with "dev found a shiny new toy" behavior.
ww520 15y ago
One advice is to use tried and mature tech that you KNOW when have to get something done under pressure. There are enough moving parts to get something done in 4 days. Let alone introducing some "hip" untested tech in short amount of time.

105 points by TheSmoke 15y ago

kennu 15y ago

So he did zero load testing before publishing an app live on television, developed with technology completely new to him?

Jgrubb 15y ago
Developed in four days, don't forget that part.
- kennu 15y ago
  It just sounds like a huge gamble. It would be quite lucky to get everything working by chance on the first try, without any bottlenecks in any part of the newly created config.
  - GFischer 15y ago
    I guess he took away some good lessons from this, since his "morals of the story" say: * Do not accept responsibility for anything that you had to do in a very limited time. * Do not accept the job if the timeline is short, the work is big and the load is heavy. * Load test your application no matter what the cost. If they want to get them all, they need to pay for them all. Failing is hard, but I'm sure he learned :)
    - gbog 15y ago
      I'd add "don't ever change your techno 4 days before you go on TV"
skormos 15y ago
Seriously, the article should've been titled "Why You Can't Deploy Publicly Available Apps in Four Days".
sylvinus 15y ago
yes to me this is the craziest part.

samarudge 15y ago

Interesting, I recently worked on Project4Awesome, a community event on Youtube. Our site used a single MongoDB node on a relatively standard server and handled >100k pageviews in 24 hours with no caching. That being said, I've worked with Mongo before and know many of it's issues, so I knew what to expect. Though, without meaning to criticize the author too much, increasing various ulimits should be standard on pretty much any database server, I would have thought?

po 15y ago
It is hard to tell but it seems like he's talking about serving 1/3 of your daily traffic in a few minutes. When web servers fail, they tend to fall off a cliff.
- lrobb 15y ago
  Yeah, I read it as 30k requests in a matter of seconds.
- samarudge 15y ago
  Indeed, but Mongo is actually quite good at failing predictably. I think the biggest issue is not Mongo, but failing to test the application correctly. Particularly when working with a new technology. Mongo isn't the best fit for everything but nearly all the issues he describes are more down to not having enough time to plan and test before the project, and one would have probably encountered similar issues if using nearly any software at that scale, with no prior knowledge or tuning.
kennu 15y ago
Do you have any advice on the MongoDB issues to be aware of in this kind of usage? I wonder if there is a checklist somewhere.
- DanielShir 15y ago
  I'd also appreciate the same checklist samarudge :)
- 15y ago

emmapersky 15y ago

I thought MongoDB was webscale. hmmm. ;)

dasil003 15y ago
Yes but apparently not televisionscale.

EwanToo 15y ago

Given that his disclaimer reads: DISCLAIMER You are about to read a long story on how I got burnt with MongoDB and depressed with it. I am not blaming MongoDB, anyone using, advocating or developing it. I am blaming myself for this. MongoDB is a good tool. You can use it but just make sure it is what you need and it handles your requirements very well. This is not specific to MongoDB but applies to every tool we use. I think it's a bit harsh for people to criticise either him or MongoDB too much. At the end of the day, it was a big rush job, no time was allowed for testing for scaling, and it fell over. My only real surprise is that it wasn't Apache that fell over first.

davidw 15y ago
> My only real surprise is that it wasn't Apache that fell over first. The Apache web server is a very well-tested, tried and true piece of infrastructure, even if it's not "hip" these days.
- dasil003 15y ago
  True, but it often needs tuning to handle massive loads.
- EwanToo 15y ago
  True, but it's defaults are not really "battle ready" and since they didn't tweak MongoDB in advance, I can only guess that Apache was already pre-configured for that load - perhaps they used it for their existing web servers?
  - spydum 15y ago
    I would guess Apache httpd is a tool they are familiar with and comfortable managing. Trying to building a high performance MongoDB system in just a couple days was probably a bad first project with the tool. They really should have just stuck to what they know, until familiar with the new shiny.
TheSmoke 15y ago
apache had not fallen first because netscaler is load balancing between 26 web servers and they are already pre-configured for all these heavy load.

psyren 15y ago

If you have 4 days to implement something important, the last thing you should do is pick a technology you have no experience with.

meanguy 15y ago

"the only thing I had in my mind was MongoDB." Well, when you approach it like that...

jeswin 15y ago

Among reasons, I see stuff like this: "Appearantly someone set the connection timeout to 9000 secs which means a connection to a MongoDB instance through Netscaler is open for 2.5 hours. I have set it to 20 secs. I am not looking for someone to blame but probably it's our sys admin. Wouldn't any database give you plain bad performance if settings were badly configured? I understand it was merely one of the reasons, but sometimes all you need is one.

TheSmoke 15y ago
this was a reason specific to our stupidity. why on earth would someone keep connections open for 2.5 hours? how he misses this setting? my point in this is, if you have something that balance connections between servers, check if it kills them or not.
- hieronymusN 15y ago
  Were you doing any connection pooling with Mongo? I'm curious what driver you were using, etc.
  - alexchamberlain 15y ago
    He gives the impression in the article that it was PHP.
- mnutt 15y ago
  Perhaps it was just some mismatch between the expectation (pooled, long-running connections) and reality. (many, many short-lived connections) I think another takeaway from this is any time you need to adjust the ulimit, take a step back and ask "Does this make sense? Is my use case exceptional enough that it won't perform within the default limits set by my OS?"

chris123 15y ago

Sounds like less of a Mongo problem and more of a "trying to do too much, too fast, too big" problem. The Greek call this "Hubris." Great lesson/reminder to share. Thanks.

wseymour 15y ago

Stop blogging and start growing a pair. Aside from the monumental error of judgement in developing on an unfamiliar database in 4 days with zero testing, the 'morals of the story' crack me up: "Do not accept responsibility for anything that you had to do in a very limited time." Yeah, because the ability to stand by your decision is a function of deadlines. "Do not accept the job if the timeline is short, the work is big and the load is heavy." Wow, that's some pretty serious ambition right there. I also like the way these points are first on the list, before actual lessons like: "Do not have trust in any of your tools until they prove themselves."

GFischer 15y ago
I think that learning to stand up to people making unreasonable requests is an important lesson. I hope he also learned from his error in judgement. And he actually did have some courage in submitting his mistakes in public for us to discuss :)
- wseymour 15y ago
  +1. I should change that opening to "stop blogging and continue growing a pair."
- TheSmoke 15y ago
  i have written all these because i have made these mistakes so someone in a similar situation does not. it's not about courage, it's knowing yourself. when you do stupid things, you have to acknowledge so you don't do them the very next time. since the show has 5 more weeks and we will have more load, more requests, more everything, i would like to hear everyone's opinion on that.
dextorious 15y ago
""" "Do not accept responsibility for anything that you had to do in a very limited time." Yeah, because the ability to stand by your decision is a function of deadlines. """ It most certainly is. One can be forced to come up with something under a tight deadline that he wouldn't under normal circumstances.

level09 15y ago

This post was very helpful indeed. we are facing almost simliar issues with our default box configurations (ulimit and connection counts etc) while we are serving real time updates to our visitors.. Thanks for posting this.

rtperson 15y ago

> We could not test the application because we were told about it on tuesday and it had to be ready on sunday. Well there's your problem right there. That is not an appropriate timeframe for a new app. The only professional response to such a request is: "It cannot be done. Not if you want this app to work in production. Here's why." Follow up with an explanation of why QA and load testing are crucial. If they press you, make your next follow-up a letter of resignation.

haasted 15y ago
Also, if you accept the timeframe, go with only known technology. This is not a scenario for getting experience in a new platforms.

rjurney 15y ago

Bigger machines. More RAM. When in doubt, and in a rush, throw hardware at the problem.

jm4 15y ago

I don't think the author is exactly unnecessarily bashing MongoDB, but MongoDB is not the problem here. It is only part of the story because that's what the author chose to use. I would think that under these circumstances and time constraints, any solution the author might have come up with would have failed. The bottom line is there were only 4 days to go from idea to supporting 30k concurrent users. That is a recipe for disaster. There are so many things to consider and implement in addition to the software. Nevermind the fact that there were probably laughable requirements for this project and no time allocated to design and testing. There wasn't even sufficient hardware to support the application. This project was a failure almost from the moment it was conceived. This guy never even had a chance. There is a story in here, but it is not so much about MongoDB.

TheSmoke 15y ago
as i have said in the disclaimer, i'm definitely not bashing or blaming mongodb for anything. when you read it, you will see that i have done a 30K/sec load testing. the url it was pointed at was doing writes to mongodb. it opened every connection, wrote to the db and closed the connection. this is a huge success for mongodb. the problem is, when you send every single read query to mongodb under a heavy load, within a few seconds, it will just accept connections and will not respond to queries. this is definitely not the case for writing. writing handles them very well. i really would like to know how we screwed there.
- jm4 15y ago
  I don't know enough about MongoDB to answer your question, and it sounds like you didn't know enough about it to even consider putting it into production for this project. That's where you got screwed. Trying out new technology when you have a likely impossible deadline and then having it fall over in miserable fashion for everyone to see is a good way to get fired. At that point, the debate is not over the ridiculous constraints placed upon you, but rather over the decisions you made. http://www.mongodb.org/display/DOCS/Connections http://www.mongodb.org/display/DOCS/Connections That page says each connection spawns a thread. That could be a good lead. The documentation suggests using a connection pool (although many drivers will do this automatically). Were you using one? http://stackoverflow.com/questions/8439639/mongodb-max-connections http://stackoverflow.com/questions/8439639/mongodb-max-conne... That StackOverflow question is about supporting 20k concurrent users. It could also be a matter of insufficient hardware. I don't know, but I would start looking at what kind of performance can be expected in different setups and go from there. The most important thing is to properly test your setup. Prepare for a process of trial and error and expect it to take some time.

benmmurphy 15y ago

how many db servers can handle 20k connections. dbs usually aren't written to handle that. most mysql implementations are going to have problems > 500.

tmountain 15y ago

I work at a company where we use Mongo to store and retrieve session information for over 2.5 million users per day. Our initial deployment crashed and burned in a similar fashion for one very simple reason. Mongo opens a thread per connection. Under this model, you simply cannot point every Apache request to Mongo and expect it to handle the load. The solution is fairly simple. You can either use persistent connections at the driver level (the PHP driver provides this, I'm not sure which other languages do), or you can put mongos (used for sharding) in front of mongod. Mongos does connection pooling and tremendously reduces the load on your mongod server. The author of the post mentions that redis is "heaven sent". Well, redis handles its connections in an asynchronous fashion which is why it can handle a large amount of direct traffic gracefully. I believe there are plans to change mongo's connection handling to use select/epoll, but I'm not sure when.

hello_moto 15y ago

I probably don't like the tone of my comment on this one but TheSmoke, please don't do this repeatedly. ever. again. While I respect your honesty, but after reading this particular blog post and a few bits of your website, I think you should slow down and value tried and tested tools. Let's start with your "About me:" "Mengu, is a web developer with Ruby on Rails, Grails, Django, Pyramid, web2py, CodeIgniter and jQuery in his tool box. Available for any kind of web development jobs. Read more.." Lots of technology there. Then you take on a project where you chose MongoDB. I think I'm starting to notice a trend here. You burned your team member for not standing up for your choice. "Our manager accused me and MongoDB for the failure while we all have agreed to use MongoDB in the team. None of my teammates stood up." My prejudice sense something is off: I felt that you were the one who pushed MongoDB hard to the point where your team member gave up and just "alright, let's just get this ball rolling since we have 4 days anyway..." rather than argue with you. It fails. You got blamed alone. None of your team member stood up for your "ignorance" of tried-and-tested tools. That's fitting if I may say so. Don't do that in public blog. Don't ever ever do that. Especially when it's your fault. You also started off with saying that "MongoDB is okay" then trash the community and not willing to purchase for the support while admitting that you do need the support. I know it has been hard for you but you really need to step back a little bit and do a retrospective on your decision-making ability. I know I'm saying harsh thinsg but I hope you understand that there's a bigger problem and MongoDB is not that problem (at least not now). In my short time in this hi-tech world, I've met and worked with a few people who religiously pushing new hip tools with zero experience (using the said tools) lately (especially lately, due to the explosion of blogs and hip tools) and would want to spread the blame to everybody else if it fails just because they agreed to use it. It's a common trend that most managers can see quickly and it may not be good for your future career.

TheSmoke 15y ago
hi. i'm not offended by what you said. in fact, i'm glad that you took your time and wrote to me. if i was reading a post like mine, i would also come to a conclusion like yours. unfortunately, i have to say that you are wrong. our team consists of 3 developers. there is not a single minute that i have pushed mongodb or anything else hard. this is a web site that has thousands of reqs / sec. nobody in our team can push new tools let alone pushing religiously. this is not a toy for new things to try. "i" was accused because it was me who stood up for the decision, among the team. plus, i am brave enough to take the blame when i am to blame. thanks again.

jerf 15y ago

Put this in the URL bar for the page: javascript:void(document.getElementsByClassName("post-body")[0].style.whiteSpace = 'pre-wrap')

whyme 15y ago

> THE MORAL OF THIS STORY The OP missed: When given a project that has high visibility and a very short timeline, don't wing it and choose a technology that 1. you have no experience with and 2. you've done no testing on. Sorry, but I feel the OP is failing to see he made the critical error.

mrinterweb 15y ago

I appreciate that the author put the disclaimer at the top that MongoDB is not to blame, but I do feel that for someone skimming headlines, the impression they may arrive at is MongoDB = bad.

PaulHoule 15y ago

so far I haven't been impressed with mongodb. i've seen it break down and die doing tasks that mysql does without breaking a sweat.

cagenut 15y ago

This is straight up web operations 101, has nothing to do with MongoDB. Hearing you pushing for a new tool at the last minute, then changing its setup/config/architecture repeatedly over a few days, then blaming your sysadmin for a setting. Man I'd hate to have that guys job, you just steamrolled him with "dev found a shiny new toy" behavior.

ww520 15y ago

One advice is to use tried and mature tech that you KNOW when have to get something done under pressure. There are enough moving parts to get something done in 4 days. Let alone introducing some "hip" untested tech in short amount of time.