One of things what's happening is due to not understanding the technology they are using. This is especially with developers who are just "developing" and are n

One of things what's happening is due to not understanding the technology they are using. This is especially with developers who are just "developing" and are not that interested in anything outside of their domain. For example in my company, we went with Mongo because as I was told: "that's the industry standard", "unlake relational databases it let us scale and handle our huge amount of data" and also "it is schemaless so we are no longer constrained by DBAs". Side note, the schemaless part is a bit fake benefit, because the data still has a schema, but the schema is now part of your. It seems like it makes things a bit easier, but you'll pay it with interest because as you are making changes the application is becoming more complex. Also I know there are exceptions, but if your data is highly relational (like in my company) storing it in NoSQL will come back to you back with vengeance. Back to the original topic, we have several databases, we split them by function. One of them is a database that is used for geolocation (a bit more than that, but roughly matching IP to zip, or latitude/longitude to zip). I should also add that we previously used Coherence to hold that data and we keep it in RAM within 32 instances of m3.large (on AWS). Recently we decided to migrate away from Coherence to Mongo and it is considered a success because we replaced 32 of those m3.large instances with just 3 x m3.large instances. Huge savings it seems. In addition to those instances keeping the data we have an infrastructure developed to move data from our authoritative source (Postgres databases) to Coherence/Mongo instances. We use AMQP for that :) so we have nodes that are used solely as brokers (and of course we have 10 of them because a single one was crashing under the load) we have producers(that fetch data from Postgres and push as a message) we have consumers (that take the data and put to Mongo) and several other nodes to control all of that and of course that's also "industry standard". Of course we need all of that because (and this was especially true with Coherence) if something happens all data is lost and we need to "bulk-load" it. That process takes several hours because AMQP was not designed for huge binary transfers like this because it is a messaging protocol. And because we can't afford to be offline for entire day when something like this happen we have multiple environments so we can cut over to a backup one. Mongo supposedly helps here a lot, because the data is also kept on disk. It's kind of interesting to see how one decision made many years ago in the past can affect bunch of subsequent ones. Anyway, I did POC out of curiosity. My reasoning was that if we treat Postgres as the authoritative source, why not keep it in a way that is directly query-able (currently the data is stored in a way that makes it easier to bulk load it). The results were better than expected. Not only it is faster (with PostGIS and ip4r extensions every query takes fraction of millisecond to run), but also the data which was about 8GB of it, after loading it to Postgres it was only 500MB. How is that possible? Well, because Mongo (and I suspect Coherence) has just simple indexing capability, in other to have a reasonable performance, developers decided that for every row that holds IP range they would simply list all IPs in that range as 64 bit integers and placed an index on that. I would love to see what would they think of if they would have to add support for IPv6 as well[1]. I'm currently having an uphill battle to convince other teams that this is the right path. I really would like to skip the crap and go with this higher, but that's probably unwise. [1] 2^128 is more than estimated number of stars in the universe.