4 ms·
You should know basic cap theorem before reading this. If you don't, go read a 1 paragraph summary. Fyi, this has nothing to do with you. I'm using you to soap
by heretohelp 14y ago
You should know basic cap theorem before reading this. If you don't, go read a 1 paragraph summary.
Fyi, this has nothing to do with you. I'm using you to soapbox about some data/scaling misconceptions. This subject happens to be one of my foremost interests.
Riak (AP, HTTP interface) is just plain a pain to use. Don't use it unless you have very specific use-cases in mind for the BitCask backend or you know what you're doing (aka, don't have to ask open-ended questions like this, no offense). Riak is amazing at a constrained set of use-cases, and pretty awful at most other things. The vector clocks, conflict resolution, and awful AWFUL API and documentation are a goddamn atrocity. Might be cool if you really need the magical replication/clustering, but realistically Cassandra and ElasticSearch offer the same wicked-cool scaling. No multi-master replication in the community/free edition. THIS IS A MASSIVE PAIN FOR LARGE DEPLOYMENTS. Also, ripping data out of the fucker is a pain.
Hypothetically Riak allows intelligent conflict resolution. In practice, this is like getting your wounds reopened and salted with ritualistic regularity.
MongoDB is just sorta..."okay" at a variety of things and especially things that can be done with sharding. Replication in MongoDB is a joke, as is the underlying infrastructure of i t. I use it as a stand-in for what most other people use an RDBMS for. I'm generally relatively disciplined, so I haven't paid the dire DIRE costs some people pay for being unhygienic with document stores. I've seen people totally trash their data in the absence of schematic enforcement. I wouldn't recommend MongoDB except to startups that I trust to know what they're doing.
MongoDB is especially handy for discrete/isolated user data and environ as its designed to shard. I'm not really comfortable describing MongoDB as being designed for denormalization because that's not really true. My real metric for denormalized data is Hadoop/HBase/Cassandra, and MongoDB totally shits the bed after the documents get past 16mb IIRC. The limit used to be 4mb.
Oh and by the way, don't take MongoDB or Riak's "map-reduce" support seriously at all. Just don't even bother. Pretend they don't exist.
HBase and Cassandra are both more solid than MongoDB and easier to use than Riak, they're more specialized than MongoDB though.
A few things to keep in mind:
Cassandra, when it first got open sourced, was frankly awful. It's actually improved a lot, to the point where it's no longer the intense pain point for Reddit and Formspring that it once was. If you need SRSFACE replication, truth-propagation, and tuneable consistency, Cassandra is your girl.
Cassandra is nominally AP, but allows tuneability to full-blown CP by all rights with ALL (it can otherwise use QUORUM, ANY, etc.). Cassie is conceptually simpler than Riak due to using timestamps rather than vector clocks to track state transitions. Hardcore database theorists will complain about this loudly. I remain undecided.
HBase is a bit simpler, but it's built on HDFS. This is, depending on your point of view, either a great thing or an awful thing. HBase is strictly a CP wide-column store. You can pretend it's Google BigTable, but that would be a dire mistake. HBase is equivalent to BigTable like Bangladeshi slums are equivalent to the Taj Mahal. Google's stuff is way...way better and, IMHO, contributes to the design being a lot more practical. It's my opinion that modeling wide-column stores and map-reduce frameworks on top of a distributed filesystem only works if that DFS is extremely top-notch.
HDFS is extremely not top notch. I'm still waiting for someone to leak the source to GFS or Colossus. When that day comes, I will probably cry tears of joy until I die of dehydration.
If you're using HBase or Cassandra, you're using a wide-column store. Cassandra is the more flexible of the two, HBase is more well-understood. Use Cassandra if you need AP/CP tunability, otherwise use HBase. Hadoop/HBase people are easier to find anyway. I personally prefer Cassandra.
Cassandra replication is more auto-magic, HBase is less auto-magic being built on HDFS. Cassandra is thrift-only, HBase is everything you normally get with the Hadoop ecosystem. REST, Java, Thrift, etc.
Cassandra is P2P, HBase is master/slave. HBase means finagling with SPOF Zookeeper nodes and all that other contemptible HDFS bullshit. Cassandra scales better. When Facebook built messaging on HBase, they smacked right into the usual HDFS "feature" that hits everybody with a large deployment.
Basically, they had to sub-cluster and shard the fuck out of it. That's a lot of work. Cassandra hasn't yet necessitated this. This is typical for non-trivial Hadoop/HDFS deployments. It's also a massive pain.
Have to wonder how Google is faring with Colossus in comparison. Hadoop is just so goddamn awful.
Example problem that works well in a wide-column store: storing and updating the 1,515,106 followers a single twitter user has.
You can use HBase and Cassandra as general-purpose data stores, but that's not really a good idea.
Realistically by the time you need the kind of scale either can offer, you've broken down your data/ops needs into discrete problems to be solved.
It looks like this:
"We need a work queue, job dispatch, and distributed filesystem for the OLAP...a wide-column store for tracking followers...a SQL database for payment information...a high-throughput cache for denormalized projections of backend data for the frontend...a sharded index for searchable data"
Not like this:
"Well. We used (MongoDB|Riak|Cassandra|HBase|Neo4j|PostgresQL) for our data and it sprinkled scaling fairy dust on our foreheads like good little catholics on ash wednesday and now our scaling problems are solved."
I'm not taking questions unless you're in the bay area and offering beer. Read a white paper if you can't send beer wenches to my door.
- markokocic 14y agoDown voted by mistake. Happens all too often when voting on the phone. I usually don't care, but your comment was exceptionally good and didn't deserve down vote.I wish someone finally makes wider gap between arrows.