8 ms·

MongoDB vs. Clustrix Benchmark

76 points by sergei 16y ago

wmf 16y ago
Price is the elephant in the room here; NoSQL exists because people aren't willing to pay for real databases. Building yet another expensive (i.e. > $0) database that doesn't even work in the cloud won't help the Web 2.0 crowd.
- sergei 16y ago
  These folks disagree with you. There are many more behind them. http://gigaom.com/cloud/clustrix-lifts-the-curtain-on-early-database-customers/ http://gigaom.com/cloud/clustrix-lifts-the-curtain-on-early-... And plenty of folks use MySQL (and PogreSQL to a much lesser extent). You just can't scale those.
  - space-monkey 16y ago
    Last time I heard, twitter still uses MySQL for the statuses (tweets) table. They did have a plan to migrate to Cassandra, but didn't go all the way through with it. So I find it hard to agree with "you just can't scale those". It may be a lot of work, but for some applications you can scale them. Edit: twitter cassandra link (don't know if this is the latest): http://engineering.twitter.com/2010/07/cassandra-at-twitter-today.html http://engineering.twitter.com/2010/07/cassandra-at-twitter-...
    - ethangunderson 16y ago
      It's worth noting that Twitter has built a lot on top of MySQL to get to the scale they're at. Take FlockDB for example, https://github.com/twitter/flockdb https://github.com/twitter/flockdb So I guess that statement should be written as "you can't scale with just those". :)
  - JoachimSchipper 16y ago
    You can't scale MySQL/PostgreSQL? Care to expand? (Although Facebook's schema is likely an abomination flying in the face of every normalized form, it does run on MySQL. And it's not like throwing PostgreSQL on a big box doesn't go far.)
  - foobarbazetc 16y ago
    I can scale PostgresSQL pretty well, thanks. PostgreSQL is used on some of the biggest databases in the world, but no one's going to tell you which. :) Your benchmark is pretty much irrelevant because Clustrix sells hardware appliances, not the software standalone. Edit: That's not to say MongoDB didn't fail your benchmark. But the issues you highlight (single mutex) are known. Test against a real database and your results don't look that impressive for a 10 node setup. All that said, Clustrix looks great. Just make it available sans-hardware.
- mark_l_watson 16y ago
  I was going to make the same comment. Even very profitable companies are usually price sensitive so +1 for open source.
  - mjw0 16y ago
    Profitable companies have usually figured out that it's better to buy a solution that solves their problem rather than: (a) pay for engineer time to integrate a solution that isn't quite right, (b) take the opportunity cost hit while waiting for (a) to be done. These costs can pretty quickly dwarf the purchase costs of licenses and hardware. Of course if there's a free solution that just slots right in, double bonus, but for the most part the cost of hardware/software is noise next to the cost of people to maintain it.
- jbellis 16y ago
  Well, that's partly right. Yes, clustrix is more expensive than the open-source databases. But, companies like Twitter and Netflix certainly have the budget for Exadata (which is what clustrix wants to be when it grows up); they are using Cassandra instead not just because it scales on commodity hardware (the other price factor besides licensing) but also because it works across multiple datacenters which two-phase commit systems can't do no matter how big your budget is, and because its availability (failure tolerance) model is much more robust. (Many NoSQL systems don't provide these advantages either, which is why lumping all non-relational systems together is usually not helpful.)
mjw0 16y ago
Spending millions on engineer time to avoid buying a real database is not good finance either. At least with this approach you can start with the (free) MySQL and only pay once you're sure your idea has traction.
moe 16y ago
Winning a benchmark against MongoDB on a non-trivial workload is a little bit like winning the special olympics. I'd be more curious to see how Clustrix performs against Cassandra, Riak or HBase in their respective domains. Those seem to be the more serious contenders when it comes to "Big Data".
- sergei 16y ago
  I chose Mongo because it gets a lot more attention on HN than any other database. I don't remember the last time I saw a post on Cassandra on here...
  - btilly 16y ago
    I think I see Redis more than Mongo.
  - JoachimSchipper 16y ago
    I get the impression that Mongo is simple to use and simple to program for; Cassandra is quite good, but also more complicated.
- mayank 16y ago
  Would you care to elaborate? Seriously, do you have any pointers or benchmarks for a Mongo vs Cassandra vs X comparison that isn't purely anecdotal, and which takes the relative strengths of each into account in a robust way? A lot of blog posts with performance numbers seem anecdotal or loaded towards one option over the other. I'm excited by some of the underlying technology in Mongo and its ilk (gossiping protocols, etc.), but there's no doubt that they require different programming techniques than traditional RDBMS. Bit like apples and oranges, isn't it? EDIT: I noticed you changed RDBMS to "big data". I'm still curious if you have any pointers to fair benchmarks though.
  - moe 16y ago
    Well, don't trust a benchmark that you haven't faked yourself they say. I was just trying to point out that MongoDB is too easy a target here. The problems it has under high load are fairly well-known, at least to anyone who tried to benchmark it outside of their MacBooks. Just bulk-load a couple million records and watch it tip over if you don't believe me - I'm not making it up and neither is Sergei. However if Clustrix wants to impress with benchmarks then they should pick an equal opponent. MongoDB is not exactly relevant for companies that consider the calibre (and cost!) of Clustrix.
bryanmig 16y ago
So a guy who is an expert in Clustrix (and knows how to setup, tune, etc) compares it against some other technology that he does not know (and does not know how to setup, tune, etc) and comes to the surprising realization that his technology is better? Where have I seen this before? Oh right.. every time I see "Technology A vs Technology B" comparisons. Naturally his results are in his favor, otherwise he would not have posted them.
- jbellis 16y ago
  In fairness, you don't have to tune MongDB poorly (deliberately or otherwise) to get poor performance with a workload involving substantial numbers of writes; it's well-documented that there is a global lock (http://www.mongodb.org/display/DOCS/How+does+concurrency+work http://www.mongodb.org/display/DOCS/How+does+concurrency+wor...) that prevents reads during write operations. That said, there are certainly nosql systems with better scaling and concurrency stories* than mongodb out there that he could have benchmarked against. :) *I'm a cassandra committer
  - dolinsky 16y ago
    Just a point of clarification, but it is a per-server lock, not a global lock across the whole database.
psadauskas 16y ago
A blog post by the founder, without posting the benchmarks themselves? How can anyone expect to take this seriously?
- sergei 16y ago
  OK. Fair enough. I'll post the benchmarks.
  - ajays 16y ago
    You can't randomly throw numbers around without listing your benchmark code, the config files, etc. Secondly: even though it becomes clear eventually, you should mention up front your relationship with Clustrix. Just because you are a founder of Clustrix doesn't necessarily invalidate your findings, but full disclosure is always a good idea.
  - bsg75 16y ago
    Along with licensing costs?
ozataman 16y ago
Another point to notice: He seems to have run his benchmarks under very trivial data-schemas with a single/simple table. A big (speed, simplicity) advantage of No-SQL is the ability to embed lots of data within the parent model and manage a single table where you would have to manage many in a SQL database. I would be very interested to see a comparison where a large and "real" data model (that contains 6-7 "joined" tables for the SQL setup and a single table with the embedded document model for the No-SQL setup) is injected into each of the technologies. Also, it is just horrible style not to include the benchmark code for peer review. Delivers near-0 credibility.
- kchodorow 16y ago
  Excellent point. MongoDB doesn't claim to be any faster than other dbs at the simple stuff (in fact, it's often slower because we haven't had years to optimize everything). The speed gains that people usually see are because they can just fetch one document, instead of doing complex joins or aggregations.
  - cheald 16y ago
    Indeed. A lot of the reason I like document databases so much is that you can solve problems differently (and many times, much more easily) than you could in a relational database. Compare: db.posts.find({tags: {$in: ["foo", "bar"]}}) to: SELECT * from posts JOIN taggings ON taggings.post_id = posts.id JOIN tags ON tags.tagging_id = taggings.id WHERE tags.tag IN ('foo', 'bar'); (Single query tag lookup; naively joins the entire taggings and tags tables before limiting with WHERE) Or a "better" query with two subselects (yikes!) SELECT * FROM posts where post_id IN (SELECT taggings.post_id FROM taggings WHERE taggings.tag_id IN (SELECT id FROM tags WHERE tags.name IN ('foo', 'bar'))) And that's the "find where any tag matches" case. Try the "when all tags match" case ($all in MongoDB), and you'll go grey a few years earlier.
    - fedd 16y ago
      relational folks can denormalize when they want to
jhugg 16y ago
The attacks on NoSQL seem a little harsh to me. People needed scalable systems. They needed them ASAP. Nobody in the RDBMS camp was even hinting at products targeting this market. Then, the NoSQL camp built systems that scale (albeit with compromises). The fact that this has spurred the others to start building scalable RDBMSs is great. But let's not pretend these new RDBMSs won't have compromises, they'll just have different ones. The important thing is for developers to make smart decisions about what tradeoffs make the most sense for scaling their application. Different applications will require different tradeoffs. Disclaimer: I work for VoltDB.
jnewland 16y ago
has anyone out there actually used or evaluated clustrix? i haven't been able to find anything on the internet about this thing that hasn't come straight from the company.
antirez 16y ago
In this article MongoDB == NoSQL, that is not the case. Different NoSQL solutions have different use cases. Also IMHO MongoDB is pretty SQLish in the data model, so you are actually comparing two implementations of a similar data model here, and one may be superior to the other one or the other way around I guess. No surprise. A more interesting attempt is IMHO to check how the difference in the data model of some NoSQL solution can lead to very different performances. For instance Clustrix VS Redis can be interesting. Examples: 1) A lot of writes against a table where you require then to get things ordered by insertion time. With Redis is is just LPUSH + LRANGE. Try to do a read/write test where many clients are writing and reading at the same time (real world), against a table (or Redis list) with millions of elements. 2) Range queries when there are a lot of writes against this indexes. For instance a table with a score (we are modeling an online game leaders board), a lot of inserts of new scores. Get ranges between random intervals at the same time. Again, many clients writing, many reading.
richardmarr 16y ago
Sergei, I think it was a mistake to put this post on a blog with no other content. I think it leaves the reader with the impression that this is the only thing you want to contribute to the community... and as other commenters have pointed out that contribution could be perceived negatively. Something to think about next time.
- JoachimSchipper 16y ago
  Every blog has a first post. What's so bad about that?
va_coder 16y ago
First off, how do you download Clustrix? With MongoDB, simple as pie: http://www.mongodb.org/downloads http://www.mongodb.org/downloads Now, where are the docs? Again, Mongo has great docs http://wiki.mongodb.org/display/DOCS/Home http://wiki.mongodb.org/display/DOCS/Home How can I verify your claims? Oh that's right, you call a salesperson first....
- PaulHoule 16y ago
  Well, there are quite a few commercial database vendors that offer parallel and clustered RDBMS products, and many of them appear to be quite good. Unfortunately, they've got a terrible marketing problem. Before 1998 or so, a relational database was an expensive product that you got from a vendor like Oracle. Since then, a generation of people have grown up that think about using a commercial RDBMS the same way most of us think about putting our hands in a toilet. @va_coder hits the nail right on the head, it's not just the cost of the product, it's the cost of the buying process. If I want to trial a product that is open source or has an OS or free edition (that could be mysql, mongodb, postgres or even Virtoso OpenLink or SQL Server Express) I can download it, read the docs and play around with it and learn a lot in a few hours. I might learn that the product is not for me, or I might get a positive impression and feel ready to commit coding time to it. If I want to trial Oracle or Clustrix, well, I'm going to have to start a contact with a sales organization and then they need to pre-qualify me, and then they need to qualify me and then I'll spend a few hours on the phone talking to people (which might take a few weeks in wall-clock time.) Even if they give me a 30 day free trial, I could easily spend $500+ of my time just getting the trial... And once I've gotten to the point where I'm negotiating with one vendor I'm going to feel a lot of pressure (internally or from my superiors) to talk with some competitive vendors too to make sure I'm making the right decision. It's a shame because, certainly, a company like Clustrix could use the revenue they get from product sales to support an awesome development team and really deliver a better product. On the other hand, they have marketing channels that are aimed at large organizations that can afford an expensive buying experience... and it's an expensive selling process for them when they've got to do "complex sales" that require approvals from a large number of stakeholders. They've got to pass those costs onto you. The trouble with this model is that tomorrow's large organizations are today's small organizations. Today, Facebook could afford just about any commercial software that's out there. However, they made critical technology decisions (that are difficult to reverse) back when they were a little company that could only afford MySQL.
sergei 16y ago
I updated the post with the benchmark source.
- fedd 16y ago
  you updated your post and i am waiting people here at hacker news to criticize your benchmark setup. maybe i missed smth but i still saw only the suggestion for you now send your benchmark to mongodb guys. edit: please discard, now i see there they say you should include joins to your clustrix benchmark. waiting for the reply
fedd 16y ago
> Interestingly enough, we never heard that SQL or the relational model was the root of all their problems. i really think that sometimes SQL and the relational model is a problem. at least, the relational database design is a course in a university, along with (object oriented) programming. so, to properly use postgre or mysql in your sophomore web startup, you should know two things well, or have a clever db guy... i have seen brilliant web programmers that design nightmare database schemas.
- mjw0 16y ago
  So relational data is hard but implementing appropriate consistency checks in your application is easy? Or do you just skip that second part and hope for the best?
  - fedd 16y ago
    imho, the second, except that some consistency is present ib document dbs: like in the order example, all order items will be within the document and wont be lost without losing the whole order. no-one would ever use 'eventually consistent' databases for financial or trading data, i think. they're for higly scalable consumer web projects
luigi 16y ago
He talks about populating MongoDB with "rows". No one should be using (or benchmarking) a technology unless they understand what it actually is, and what problems it aims to solve.
- jdf 16y ago
  The term "rows" could certainly be replaced with "tuples", "objects", or whatever else you prefer without changing any meaning of the post. If you look at the benchmarks posted on Mongo's site http://www.mongodb.org/display/DOCS/Benchmarks http://www.mongodb.org/display/DOCS/Benchmarks you can see that most of them compare Mongo against MySQL. Certainly "rows" are being inserted into the latter.
strlen 16y ago
Pet peeve: eventual consistency isn't for scalability and performance, it's for availability. In a well designed system, the whole debate only matters during in a failure condition: a strongly consistent system gives up availability upon a certain kind of failure, an eventually consistent system gives up consistency upon a certain kind of failure. There are strongly consistent scalable "NoSQL" systems e.g., BigTable. Megastore even provides complex distributed cross row transactions. In a well tuned system, loss of availability (in a failure scenario) could be minimize to seconds. What this means for performance is that you have systems which encounter second-long latency spikes upon failures. It also isn't a binary switch: 1) Quorums can be used to achieve read-your-write consistency in the case of simple failures (loss of 1 node out of 3). 2) There's multiple kinds of relaxed consistency models. One of them is serializable consistency: you may get a stale read, but the the order of reads is the same as the order of writes. This is used by PNUTs and can be achieved by serializing the writes through a single master. This means that there's (again, for a short time period) loss of write availability in a failure, but there's no loss of read availability. 3) Paxos/multi-Paxos can be used to achieve atomic writes (all available nodes receive the write) while withstanding simple failures (similar to quorum protocols... and I believe multi-Paxos uses quorum protocols under the cover to improve liveness over "raw" Paxos). This is at the cost of higher latency (and complexity). [Edit: In this case, you're dealing with full-blown strong consistency, but with -- at the cost of latency -- the ability to tolerate certain kinds of simple failures/trivial partitions] Clustrix looks interesting, but it looks like it addresses the scalability and performance issues with RDBMS, but not the availability feature. If an RDBMS were to drop the "A" and "I" in ACID (C in "ACID" means serializable view of the execution, which is not the same as the C in "CAP": the latter means all nodes in a cluster agreeing upon what the data is which isn't required for the former), it would be possible to build a highly available, low latency RDBMS; but it would also not be as useful without atomic and isolated cross row transactions. [Disclaimer: I work on a Dynamo-style database, but fond of PNUTs as an architecture (more difficult to implement, but IMO a better fit for plurality of web applications) and generally fascinated and interested in distributed systems, databases and systems programming in general]
- sergei 16y ago
  A large part of the message I'm trying to convey is: 1. A DBMS is much more than just the interface. I'm going to write more on the subject. Whether you're using SQL, datalog, BSON, etc. -- there's a broader set of desirable features that's more important than the query language. 2. I'm not saying I agree with what the NoSQL folks are saying, or their justifications. Let's be honest: there's a prevalent sentiment that relational SQL based databases do not scale, and even further, that they somehow cannot scale. That's just not true. I saw a video the other day of some guy at Google giving a talk about the database behind the app engine. At one point someone in the audience asked about scale and SQL. His response was "Well, how well does SQL scale?" Everyone in the room laughed.
deleted 16y ago
[deleted]
drm237 16y ago
How long does it take to add a column to that table with 300MM rows? Does Clustrix require schema modifications?

76 points by sergei 16y ago

wmf 16y ago

Price is the elephant in the room here; NoSQL exists because people aren't willing to pay for real databases. Building yet another expensive (i.e. > $0) database that doesn't even work in the cloud won't help the Web 2.0 crowd.

sergei 16y ago
These folks disagree with you. There are many more behind them. http://gigaom.com/cloud/clustrix-lifts-the-curtain-on-early-database-customers/ http://gigaom.com/cloud/clustrix-lifts-the-curtain-on-early-... And plenty of folks use MySQL (and PogreSQL to a much lesser extent). You just can't scale those.
- space-monkey 16y ago
  Last time I heard, twitter still uses MySQL for the statuses (tweets) table. They did have a plan to migrate to Cassandra, but didn't go all the way through with it. So I find it hard to agree with "you just can't scale those". It may be a lot of work, but for some applications you can scale them. Edit: twitter cassandra link (don't know if this is the latest): http://engineering.twitter.com/2010/07/cassandra-at-twitter-today.html http://engineering.twitter.com/2010/07/cassandra-at-twitter-...
  - ethangunderson 16y ago
    It's worth noting that Twitter has built a lot on top of MySQL to get to the scale they're at. Take FlockDB for example, https://github.com/twitter/flockdb https://github.com/twitter/flockdb So I guess that statement should be written as "you can't scale with just those". :)
- JoachimSchipper 16y ago
  You can't scale MySQL/PostgreSQL? Care to expand? (Although Facebook's schema is likely an abomination flying in the face of every normalized form, it does run on MySQL. And it's not like throwing PostgreSQL on a big box doesn't go far.)
- foobarbazetc 16y ago
  I can scale PostgresSQL pretty well, thanks. PostgreSQL is used on some of the biggest databases in the world, but no one's going to tell you which. :) Your benchmark is pretty much irrelevant because Clustrix sells hardware appliances, not the software standalone. Edit: That's not to say MongoDB didn't fail your benchmark. But the issues you highlight (single mutex) are known. Test against a real database and your results don't look that impressive for a 10 node setup. All that said, Clustrix looks great. Just make it available sans-hardware.
mark_l_watson 16y ago
I was going to make the same comment. Even very profitable companies are usually price sensitive so +1 for open source.
- mjw0 16y ago
  Profitable companies have usually figured out that it's better to buy a solution that solves their problem rather than: (a) pay for engineer time to integrate a solution that isn't quite right, (b) take the opportunity cost hit while waiting for (a) to be done. These costs can pretty quickly dwarf the purchase costs of licenses and hardware. Of course if there's a free solution that just slots right in, double bonus, but for the most part the cost of hardware/software is noise next to the cost of people to maintain it.
jbellis 16y ago
Well, that's partly right. Yes, clustrix is more expensive than the open-source databases. But, companies like Twitter and Netflix certainly have the budget for Exadata (which is what clustrix wants to be when it grows up); they are using Cassandra instead not just because it scales on commodity hardware (the other price factor besides licensing) but also because it works across multiple datacenters which two-phase commit systems can't do no matter how big your budget is, and because its availability (failure tolerance) model is much more robust. (Many NoSQL systems don't provide these advantages either, which is why lumping all non-relational systems together is usually not helpful.)

mjw0 16y ago

Spending millions on engineer time to avoid buying a real database is not good finance either. At least with this approach you can start with the (free) MySQL and only pay once you're sure your idea has traction.

moe 16y ago

Winning a benchmark against MongoDB on a non-trivial workload is a little bit like winning the special olympics. I'd be more curious to see how Clustrix performs against Cassandra, Riak or HBase in their respective domains. Those seem to be the more serious contenders when it comes to "Big Data".

sergei 16y ago
I chose Mongo because it gets a lot more attention on HN than any other database. I don't remember the last time I saw a post on Cassandra on here...
- btilly 16y ago
  I think I see Redis more than Mongo.
- JoachimSchipper 16y ago
  I get the impression that Mongo is simple to use and simple to program for; Cassandra is quite good, but also more complicated.
mayank 16y ago
Would you care to elaborate? Seriously, do you have any pointers or benchmarks for a Mongo vs Cassandra vs X comparison that isn't purely anecdotal, and which takes the relative strengths of each into account in a robust way? A lot of blog posts with performance numbers seem anecdotal or loaded towards one option over the other. I'm excited by some of the underlying technology in Mongo and its ilk (gossiping protocols, etc.), but there's no doubt that they require different programming techniques than traditional RDBMS. Bit like apples and oranges, isn't it? EDIT: I noticed you changed RDBMS to "big data". I'm still curious if you have any pointers to fair benchmarks though.
- moe 16y ago
  Well, don't trust a benchmark that you haven't faked yourself they say. I was just trying to point out that MongoDB is too easy a target here. The problems it has under high load are fairly well-known, at least to anyone who tried to benchmark it outside of their MacBooks. Just bulk-load a couple million records and watch it tip over if you don't believe me - I'm not making it up and neither is Sergei. However if Clustrix wants to impress with benchmarks then they should pick an equal opponent. MongoDB is not exactly relevant for companies that consider the calibre (and cost!) of Clustrix.

bryanmig 16y ago

So a guy who is an expert in Clustrix (and knows how to setup, tune, etc) compares it against some other technology that he does not know (and does not know how to setup, tune, etc) and comes to the surprising realization that his technology is better? Where have I seen this before? Oh right.. every time I see "Technology A vs Technology B" comparisons. Naturally his results are in his favor, otherwise he would not have posted them.

jbellis 16y ago
In fairness, you don't have to tune MongDB poorly (deliberately or otherwise) to get poor performance with a workload involving substantial numbers of writes; it's well-documented that there is a global lock (http://www.mongodb.org/display/DOCS/How+does+concurrency+work http://www.mongodb.org/display/DOCS/How+does+concurrency+wor...) that prevents reads during write operations. That said, there are certainly nosql systems with better scaling and concurrency stories* than mongodb out there that he could have benchmarked against. :) *I'm a cassandra committer
- dolinsky 16y ago
  Just a point of clarification, but it is a per-server lock, not a global lock across the whole database.

psadauskas 16y ago

A blog post by the founder, without posting the benchmarks themselves? How can anyone expect to take this seriously?

sergei 16y ago
OK. Fair enough. I'll post the benchmarks.
- ajays 16y ago
  You can't randomly throw numbers around without listing your benchmark code, the config files, etc. Secondly: even though it becomes clear eventually, you should mention up front your relationship with Clustrix. Just because you are a founder of Clustrix doesn't necessarily invalidate your findings, but full disclosure is always a good idea.
- bsg75 16y ago
  Along with licensing costs?

ozataman 16y ago

Another point to notice: He seems to have run his benchmarks under very trivial data-schemas with a single/simple table. A big (speed, simplicity) advantage of No-SQL is the ability to embed lots of data within the parent model and manage a single table where you would have to manage many in a SQL database. I would be very interested to see a comparison where a large and "real" data model (that contains 6-7 "joined" tables for the SQL setup and a single table with the embedded document model for the No-SQL setup) is injected into each of the technologies. Also, it is just horrible style not to include the benchmark code for peer review. Delivers near-0 credibility.

kchodorow 16y ago
Excellent point. MongoDB doesn't claim to be any faster than other dbs at the simple stuff (in fact, it's often slower because we haven't had years to optimize everything). The speed gains that people usually see are because they can just fetch one document, instead of doing complex joins or aggregations.
- cheald 16y ago
  Indeed. A lot of the reason I like document databases so much is that you can solve problems differently (and many times, much more easily) than you could in a relational database. Compare: db.posts.find({tags: {$in: ["foo", "bar"]}}) to: SELECT * from posts JOIN taggings ON taggings.post_id = posts.id JOIN tags ON tags.tagging_id = taggings.id WHERE tags.tag IN ('foo', 'bar'); (Single query tag lookup; naively joins the entire taggings and tags tables before limiting with WHERE) Or a "better" query with two subselects (yikes!) SELECT * FROM posts where post_id IN (SELECT taggings.post_id FROM taggings WHERE taggings.tag_id IN (SELECT id FROM tags WHERE tags.name IN ('foo', 'bar'))) And that's the "find where any tag matches" case. Try the "when all tags match" case ($all in MongoDB), and you'll go grey a few years earlier.
  - fedd 16y ago
    relational folks can denormalize when they want to

jhugg 16y ago

The attacks on NoSQL seem a little harsh to me. People needed scalable systems. They needed them ASAP. Nobody in the RDBMS camp was even hinting at products targeting this market. Then, the NoSQL camp built systems that scale (albeit with compromises). The fact that this has spurred the others to start building scalable RDBMSs is great. But let's not pretend these new RDBMSs won't have compromises, they'll just have different ones. The important thing is for developers to make smart decisions about what tradeoffs make the most sense for scaling their application. Different applications will require different tradeoffs. Disclaimer: I work for VoltDB.

jnewland 16y ago

has anyone out there actually used or evaluated clustrix? i haven't been able to find anything on the internet about this thing that hasn't come straight from the company.

antirez 16y ago

In this article MongoDB == NoSQL, that is not the case. Different NoSQL solutions have different use cases. Also IMHO MongoDB is pretty SQLish in the data model, so you are actually comparing two implementations of a similar data model here, and one may be superior to the other one or the other way around I guess. No surprise. A more interesting attempt is IMHO to check how the difference in the data model of some NoSQL solution can lead to very different performances. For instance Clustrix VS Redis can be interesting. Examples: 1) A lot of writes against a table where you require then to get things ordered by insertion time. With Redis is is just LPUSH + LRANGE. Try to do a read/write test where many clients are writing and reading at the same time (real world), against a table (or Redis list) with millions of elements. 2) Range queries when there are a lot of writes against this indexes. For instance a table with a score (we are modeling an online game leaders board), a lot of inserts of new scores. Get ranges between random intervals at the same time. Again, many clients writing, many reading.

richardmarr 16y ago

Sergei, I think it was a mistake to put this post on a blog with no other content. I think it leaves the reader with the impression that this is the only thing you want to contribute to the community... and as other commenters have pointed out that contribution could be perceived negatively. Something to think about next time.

JoachimSchipper 16y ago
Every blog has a first post. What's so bad about that?

va_coder 16y ago

First off, how do you download Clustrix? With MongoDB, simple as pie: http://www.mongodb.org/downloads http://www.mongodb.org/downloads Now, where are the docs? Again, Mongo has great docs http://wiki.mongodb.org/display/DOCS/Home http://wiki.mongodb.org/display/DOCS/Home How can I verify your claims? Oh that's right, you call a salesperson first....

PaulHoule 16y ago
Well, there are quite a few commercial database vendors that offer parallel and clustered RDBMS products, and many of them appear to be quite good. Unfortunately, they've got a terrible marketing problem. Before 1998 or so, a relational database was an expensive product that you got from a vendor like Oracle. Since then, a generation of people have grown up that think about using a commercial RDBMS the same way most of us think about putting our hands in a toilet. @va_coder hits the nail right on the head, it's not just the cost of the product, it's the cost of the buying process. If I want to trial a product that is open source or has an OS or free edition (that could be mysql, mongodb, postgres or even Virtoso OpenLink or SQL Server Express) I can download it, read the docs and play around with it and learn a lot in a few hours. I might learn that the product is not for me, or I might get a positive impression and feel ready to commit coding time to it. If I want to trial Oracle or Clustrix, well, I'm going to have to start a contact with a sales organization and then they need to pre-qualify me, and then they need to qualify me and then I'll spend a few hours on the phone talking to people (which might take a few weeks in wall-clock time.) Even if they give me a 30 day free trial, I could easily spend $500+ of my time just getting the trial... And once I've gotten to the point where I'm negotiating with one vendor I'm going to feel a lot of pressure (internally or from my superiors) to talk with some competitive vendors too to make sure I'm making the right decision. It's a shame because, certainly, a company like Clustrix could use the revenue they get from product sales to support an awesome development team and really deliver a better product. On the other hand, they have marketing channels that are aimed at large organizations that can afford an expensive buying experience... and it's an expensive selling process for them when they've got to do "complex sales" that require approvals from a large number of stakeholders. They've got to pass those costs onto you. The trouble with this model is that tomorrow's large organizations are today's small organizations. Today, Facebook could afford just about any commercial software that's out there. However, they made critical technology decisions (that are difficult to reverse) back when they were a little company that could only afford MySQL.

sergei 16y ago

I updated the post with the benchmark source.

fedd 16y ago
you updated your post and i am waiting people here at hacker news to criticize your benchmark setup. maybe i missed smth but i still saw only the suggestion for you now send your benchmark to mongodb guys. edit: please discard, now i see there they say you should include joins to your clustrix benchmark. waiting for the reply

fedd 16y ago

> Interestingly enough, we never heard that SQL or the relational model was the root of all their problems. i really think that sometimes SQL and the relational model is a problem. at least, the relational database design is a course in a university, along with (object oriented) programming. so, to properly use postgre or mysql in your sophomore web startup, you should know two things well, or have a clever db guy... i have seen brilliant web programmers that design nightmare database schemas.

mjw0 16y ago
So relational data is hard but implementing appropriate consistency checks in your application is easy? Or do you just skip that second part and hope for the best?
- fedd 16y ago
  imho, the second, except that some consistency is present ib document dbs: like in the order example, all order items will be within the document and wont be lost without losing the whole order. no-one would ever use 'eventually consistent' databases for financial or trading data, i think. they're for higly scalable consumer web projects

luigi 16y ago

He talks about populating MongoDB with "rows". No one should be using (or benchmarking) a technology unless they understand what it actually is, and what problems it aims to solve.

jdf 16y ago
The term "rows" could certainly be replaced with "tuples", "objects", or whatever else you prefer without changing any meaning of the post. If you look at the benchmarks posted on Mongo's site http://www.mongodb.org/display/DOCS/Benchmarks http://www.mongodb.org/display/DOCS/Benchmarks you can see that most of them compare Mongo against MySQL. Certainly "rows" are being inserted into the latter.

strlen 16y ago

Pet peeve: eventual consistency isn't for scalability and performance, it's for availability. In a well designed system, the whole debate only matters during in a failure condition: a strongly consistent system gives up availability upon a certain kind of failure, an eventually consistent system gives up consistency upon a certain kind of failure. There are strongly consistent scalable "NoSQL" systems e.g., BigTable. Megastore even provides complex distributed cross row transactions. In a well tuned system, loss of availability (in a failure scenario) could be minimize to seconds. What this means for performance is that you have systems which encounter second-long latency spikes upon failures. It also isn't a binary switch: 1) Quorums can be used to achieve read-your-write consistency in the case of simple failures (loss of 1 node out of 3). 2) There's multiple kinds of relaxed consistency models. One of them is serializable consistency: you may get a stale read, but the the order of reads is the same as the order of writes. This is used by PNUTs and can be achieved by serializing the writes through a single master. This means that there's (again, for a short time period) loss of write availability in a failure, but there's no loss of read availability. 3) Paxos/multi-Paxos can be used to achieve atomic writes (all available nodes receive the write) while withstanding simple failures (similar to quorum protocols... and I believe multi-Paxos uses quorum protocols under the cover to improve liveness over "raw" Paxos). This is at the cost of higher latency (and complexity). [Edit: In this case, you're dealing with full-blown strong consistency, but with -- at the cost of latency -- the ability to tolerate certain kinds of simple failures/trivial partitions] Clustrix looks interesting, but it looks like it addresses the scalability and performance issues with RDBMS, but not the availability feature. If an RDBMS were to drop the "A" and "I" in ACID (C in "ACID" means serializable view of the execution, which is not the same as the C in "CAP": the latter means all nodes in a cluster agreeing upon what the data is which isn't required for the former), it would be possible to build a highly available, low latency RDBMS; but it would also not be as useful without atomic and isolated cross row transactions. [Disclaimer: I work on a Dynamo-style database, but fond of PNUTs as an architecture (more difficult to implement, but IMO a better fit for plurality of web applications) and generally fascinated and interested in distributed systems, databases and systems programming in general]

sergei 16y ago
A large part of the message I'm trying to convey is: 1. A DBMS is much more than just the interface. I'm going to write more on the subject. Whether you're using SQL, datalog, BSON, etc. -- there's a broader set of desirable features that's more important than the query language. 2. I'm not saying I agree with what the NoSQL folks are saying, or their justifications. Let's be honest: there's a prevalent sentiment that relational SQL based databases do not scale, and even further, that they somehow cannot scale. That's just not true. I saw a video the other day of some guy at Google giving a talk about the database behind the app engine. At one point someone in the audience asked about scale and SQL. His response was "Well, how well does SQL scale?" Everyone in the room laughed.

deleted 16y ago

[deleted]

drm237 16y ago

How long does it take to add a column to that table with 300MM rows? Does Clustrix require schema modifications?