4 ms·

Whenever I see the "You don't need Mongo DB, use an SQL database" and then in the flaming back and forth, I never see my key problem mentioned: MongoDB makes i

Whenever I see the "You don't need Mongo DB, use an SQL database" and then in the flaming back and forth, I never see my key problem mentioned: MongoDB makes it easy to scale out (replica sets and sharding), where is the "easy to setup replicated and sharded open source SQL database?" I mean, I know that Postgres has replication (via Slony? honestly, it's been awhile since I looked at their solutions) but I don't recall it being as dead simple to set up. For me, setting up replication needs to be easy because we redistribute the store as part of our product and we need scalability (both replication for redundancy and sharding for scaling). So I'm honestly asking here, where is the easy to use sharded and replicated open source SQL store that I've been missing?

AlisdairO 13y ago
Postgres has hot standby built in nowadays, and it works well. Sharding certainly isn't as easy - the technical compromises that mongo makes make it pretty trivial to implement, whereas it's relatively hard to make it work in an RDBMS while maintaining all the expected capabilities. It generally requires some application-level work on open source dbs. With that said, I really think many people grossly underestimate the effectiveness of scale-up. It's worth remembering that Stack Overflow (for example) is still running on a single pair of master/hot standby database machines.
- threeseed 13y ago
  Standby is a pretty poor solution compared to replica set let alone what Cassandra has to offer. Sharding is trivial on MongoDB/Cassandra and it is open source. So let's be accurate here. It is a problem inherent with the SQL databases. And I think you underestimate the benefits of scaling out. If I want to ensure close to 100% uptime or have a server closer to my users than Cassandra or even MongoDB would be infinitely easier to setup and manage than Postgres. These are "very nice to haves" for even the tiniest startup.
  - craigching 13y ago
    Thanks, that's very much what I'm talking about. Cassandra would be my ideal store, I absolutely love it except for the ability to index across nodes. My understanding, when last I looked at it, was that indexes were only local and didn't span Cassandra nodes. Does Cassandra now have properly distributed indexes? When I was looking at implementing Cassandra instead of Mongo DB, it seemed like we had to create reverse column family (IIRC, been away from Cassandra for a bit now). Is that still the case?
    - crypto5 13y ago
      Just curious: did you get any real problem with local indexes? For me it works just fine.
      - craigching 13y ago
        How do you use a local index when your data is distributed across numerous nodes? Maybe I'm missing something fundamental, so I'd definitely like to understand.
        crypto5 13y ago
        It's hidden from me behind client library API(astyanax in my case), I shouldn't know anything about index locality. I just send request(give me records for this index value), and get response and don't care if it's local index or distributed index. Astyanax takes care about everything, it queries all nodes.
        craigching 13y ago
        I will check that out, thanks! I was looking at Hector and Pelops at the time and I don't believe they provided anything like that. But now that you mention it, I bet Hive could do what you describe as well. Need to dig a bit deeper now!
        craigching 13y ago
nasalgoat 13y ago
That "easy to scale out" is a misnomer. Replica sets and sharding work in the technical sense, but the implementation isn't anywhere near what I would qualify as production ready. For example, today my entire production MongoDB database was running 3x slower because a single replica in one shard was down, and their buggy PHP driver kept trying to talk to it despite it being marked down. I really enjoyed waking up at 2am to deal with that. It relates back to the "easy to use" nature of their marketing. It really is super easy to use and develop on, but the minute you need to do anything important or serious, it breaks down. You aren't doing yourself any favors going with it except as a proof-of-concept.
- craigching 13y ago
  But Mongo DB being buggy isn't a reason to need to use an SQL database vs. a NoSQL store. An SQL database could be buggy as well (I still use Postgres and comparing anything to that quality-wise is just going to bring sorrow for the thing you compare it to ;) ). FWIW, it's been spotless for us so far. Our needs aren't web scale, but they're big enough to need scaling features.
  - nasalgoat 13y ago
    The main issue with MongoDB is that it's so easy to use and seems like it scales, but soon you're invested in it to the point of refactoring being a serious engineering effort, and you're stuck with something that doesn't actually offer real scaling features. So, it's less "nosql vs. sql" and more just "don't use mongodb".

AlisdairO 13y ago

Postgres has hot standby built in nowadays, and it works well. Sharding certainly isn't as easy - the technical compromises that mongo makes make it pretty trivial to implement, whereas it's relatively hard to make it work in an RDBMS while maintaining all the expected capabilities. It generally requires some application-level work on open source dbs. With that said, I really think many people grossly underestimate the effectiveness of scale-up. It's worth remembering that Stack Overflow (for example) is still running on a single pair of master/hot standby database machines.

threeseed 13y ago
Standby is a pretty poor solution compared to replica set let alone what Cassandra has to offer. Sharding is trivial on MongoDB/Cassandra and it is open source. So let's be accurate here. It is a problem inherent with the SQL databases. And I think you underestimate the benefits of scaling out. If I want to ensure close to 100% uptime or have a server closer to my users than Cassandra or even MongoDB would be infinitely easier to setup and manage than Postgres. These are "very nice to haves" for even the tiniest startup.
- craigching 13y ago
  Thanks, that's very much what I'm talking about. Cassandra would be my ideal store, I absolutely love it except for the ability to index across nodes. My understanding, when last I looked at it, was that indexes were only local and didn't span Cassandra nodes. Does Cassandra now have properly distributed indexes? When I was looking at implementing Cassandra instead of Mongo DB, it seemed like we had to create reverse column family (IIRC, been away from Cassandra for a bit now). Is that still the case?
  - crypto5 13y ago
    Just curious: did you get any real problem with local indexes? For me it works just fine.
    - craigching 13y ago
      How do you use a local index when your data is distributed across numerous nodes? Maybe I'm missing something fundamental, so I'd definitely like to understand.
      - crypto5 13y ago
        It's hidden from me behind client library API(astyanax in my case), I shouldn't know anything about index locality. I just send request(give me records for this index value), and get response and don't care if it's local index or distributed index. Astyanax takes care about everything, it queries all nodes.
        craigching 13y ago
        I will check that out, thanks! I was looking at Hector and Pelops at the time and I don't believe they provided anything like that. But now that you mention it, I bet Hive could do what you describe as well. Need to dig a bit deeper now!
        craigching 13y ago

nasalgoat 13y ago

That "easy to scale out" is a misnomer. Replica sets and sharding work in the technical sense, but the implementation isn't anywhere near what I would qualify as production ready. For example, today my entire production MongoDB database was running 3x slower because a single replica in one shard was down, and their buggy PHP driver kept trying to talk to it despite it being marked down. I really enjoyed waking up at 2am to deal with that. It relates back to the "easy to use" nature of their marketing. It really is super easy to use and develop on, but the minute you need to do anything important or serious, it breaks down. You aren't doing yourself any favors going with it except as a proof-of-concept.

craigching 13y ago
But Mongo DB being buggy isn't a reason to need to use an SQL database vs. a NoSQL store. An SQL database could be buggy as well (I still use Postgres and comparing anything to that quality-wise is just going to bring sorrow for the thing you compare it to ;) ). FWIW, it's been spotless for us so far. Our needs aren't web scale, but they're big enough to need scaling features.
- nasalgoat 13y ago
  The main issue with MongoDB is that it's so easy to use and seems like it scales, but soon you're invested in it to the point of refactoring being a serious engineering effort, and you're stuck with something that doesn't actually offer real scaling features. So, it's less "nosql vs. sql" and more just "don't use mongodb".