5 ms·

Thanks for this. I've heard lots of good things about MongoDB (and auto-sharding makes me drool), but I've heard rumours of problems with it too. Always cool

Thanks for this. I've heard lots of good things about MongoDB (and auto-sharding makes me drool), but I've heard rumours of problems with it too. Always cool to read something from the other side of the fence. Too bad that there's no other product out there that makes automatically sharding large amounts of data quite so easy...

megaman821 15y ago
Doesn't Riak and Cassandra make it easy? There is just no fast ad-hoc querying in those two.
- rbranson 15y ago
  Arguably, with a data set of any serious magnitude, there's no such thing as ad-hoc. As soon as an RDBMS is sharded, it loses most of it's ad-hoc powers. That being said, Riak Search allows Riak to perform ad-hoc queries and Cassandra now supports secondary indices along with a SQL-like query language called CQL. Of course, you'll have to pre-plan your indexes with either of those options.
  - schmichael 15y ago
    "As soon as an RDBMS is sharded, it loses most of it's ad-hoc powers." That's an overly broad statement I would avoid. We only need to query across our various PostgreSQL partitions for offline batch operations. Our application code has yet to feel restricted by the manual sharding. Secondary indexes - or rather: not restricting a query to a single partition (using your primary key) - is going to be expensive in any distributed system.
    - rbranson 15y ago
      You lose aggregates and joins, arguably the most powerful parts of an ad-hoc query system (business intelligence kind of stuff). Without those features, you must move a great deal of these concerns into the application layer (or whatever manage your shards), even if it's just rolling up aggregates across shards. This works, but again, at the loss of the real advantages of the database here as you've got to do it yourself. I agree with your second assertion. Secondary indexes should be avoided, but are a useful tool if absolutely needed. I've found that Cassandra's richer data model really allows them to be avoided in most scenarios.
  - jbellis 15y ago
    Cassandra's CREATE INDEX will happily run in the background against existing data. Without blocking other reads or writes, of course.
- someone13 15y ago
  Yeah, I've considered other options. Riak seems really nice, actually - I'm going to be trying that out next. I do like MongoDB's support for Windows, though. I'm also hesitant to use Cassandra, considering all the trouble that Reddit's had with it [1]. [1] http://blog.reddit.com/2010/05/reddits-may-2010-state-of-servers.html http://blog.reddit.com/2010/05/reddits-may-2010-state-of-ser...
  - jbellis 15y ago
    Reddit's initial problems were from being woefully underprovisioned, which they acknowledged. Here's what David King [ketralnis] said six months later: "Running any large website is a constant race between scaling your user base and scaling your infrastructure to support it," said David King, Lead Developer at Reddit. "Our traffic more than tripled this year, and the transparent scalability afforded to us by Apache Cassandra is in large part what allowed us to do it on our limited resources. Cassandra v0.7 represents the real-life operations lessons learned from installations like ours and provides further features like column expiration that allow us to scale even more of our infrastructure." [https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces9 https://blogs.apache.org/foundation/entry/the_apache_softwar...]
    - rbranson 15y ago
      Reddit also just uses it as a dumb K/V store and don't take advantage of the column-store model. In theory, all of the comments for a story could be stored as columns in a column family. While this would reduce the write throughput to something like ~1000/second -- for a single story -- because partition granularity is at the row level, within a row, all the columns in a CF are stored sequentially on disk, and it also means only a single node would need to be interrogated to get the data, rather than having to broadcast a massive multi-get across the entire cluster. EDIT: the column keys are also returned in sorted order, so this could be used to pre-sort the comment list by it's optimal access pattern.

megaman821 15y ago

Doesn't Riak and Cassandra make it easy? There is just no fast ad-hoc querying in those two.

rbranson 15y ago
Arguably, with a data set of any serious magnitude, there's no such thing as ad-hoc. As soon as an RDBMS is sharded, it loses most of it's ad-hoc powers. That being said, Riak Search allows Riak to perform ad-hoc queries and Cassandra now supports secondary indices along with a SQL-like query language called CQL. Of course, you'll have to pre-plan your indexes with either of those options.
- schmichael 15y ago
  "As soon as an RDBMS is sharded, it loses most of it's ad-hoc powers." That's an overly broad statement I would avoid. We only need to query across our various PostgreSQL partitions for offline batch operations. Our application code has yet to feel restricted by the manual sharding. Secondary indexes - or rather: not restricting a query to a single partition (using your primary key) - is going to be expensive in any distributed system.
  - rbranson 15y ago
    You lose aggregates and joins, arguably the most powerful parts of an ad-hoc query system (business intelligence kind of stuff). Without those features, you must move a great deal of these concerns into the application layer (or whatever manage your shards), even if it's just rolling up aggregates across shards. This works, but again, at the loss of the real advantages of the database here as you've got to do it yourself. I agree with your second assertion. Secondary indexes should be avoided, but are a useful tool if absolutely needed. I've found that Cassandra's richer data model really allows them to be avoided in most scenarios.
- jbellis 15y ago
  Cassandra's CREATE INDEX will happily run in the background against existing data. Without blocking other reads or writes, of course.
someone13 15y ago
Yeah, I've considered other options. Riak seems really nice, actually - I'm going to be trying that out next. I do like MongoDB's support for Windows, though. I'm also hesitant to use Cassandra, considering all the trouble that Reddit's had with it [1]. [1] http://blog.reddit.com/2010/05/reddits-may-2010-state-of-servers.html http://blog.reddit.com/2010/05/reddits-may-2010-state-of-ser...
- jbellis 15y ago
  Reddit's initial problems were from being woefully underprovisioned, which they acknowledged. Here's what David King [ketralnis] said six months later: "Running any large website is a constant race between scaling your user base and scaling your infrastructure to support it," said David King, Lead Developer at Reddit. "Our traffic more than tripled this year, and the transparent scalability afforded to us by Apache Cassandra is in large part what allowed us to do it on our limited resources. Cassandra v0.7 represents the real-life operations lessons learned from installations like ours and provides further features like column expiration that allow us to scale even more of our infrastructure." [https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces9 https://blogs.apache.org/foundation/entry/the_apache_softwar...]
  - rbranson 15y ago
    Reddit also just uses it as a dumb K/V store and don't take advantage of the column-store model. In theory, all of the comments for a story could be stored as columns in a column family. While this would reduce the write throughput to something like ~1000/second -- for a single story -- because partition granularity is at the row level, within a row, all the columns in a CF are stored sequentially on disk, and it also means only a single node would need to be interrogated to get the data, rather than having to broadcast a massive multi-get across the entire cluster. EDIT: the column keys are also returned in sorted order, so this could be used to pre-sort the comment list by it's optimal access pattern.