4 ms·

The reason why I mentioned Oracle in the demo is precisely because of the whole CAP debacle. So many people talk about CAP as if the SQL database was itself an

The reason why I mentioned Oracle in the demo is precisely because of the whole CAP debacle. So many people talk about CAP as if the SQL database was itself an impossibility, much less Oracle's scalable offering - but you can pick up the phone and an Oracle salesperson will sell you the physical device that you're implying can't exist. So, if somebody misquotes the CAP theorem, it's an indication to me that perhaps the thread isn't going to go anywhere. I'll point them to the place where they can read about it (the proof was where the CAP theorem was formalized, not Eric Brewer's original presentation), and move on. In this case, that wasn't fair to btilly, who just made a typo. So, as I said in the presentation and elsewhere here, if you have a hair-on-fire problem with scaling your database, and are willing to be an early customer of our new database, and would make a good reference customer in a few months, we'd love to work with you today. Otherwise, you'll have to wait till we open it up more widely.

moe 17y ago
but you can pick up the phone and an Oracle salesperson will sell you the physical device that you're implying can't exist. Well, thing is, part of your claim (if I understood it correctly) is that I just throw my data and my queries at your wonder-device and it will magically be fast and scale. You never mention any special care I must take, you even explicitly say it's not partitioning. This is decidedly not the case with what said Oracle salesperson will sell to me. The sale will normally include an expensive support-contract and probably a dedicated DB Admin. Because operating a RAC- or Exadata Cluster is, for all I know, far from trivial. You still have to make a bunch of critical decisions and tradeoffs upfront or it will probably suck. Moreover you're effectively saying that you solved what Amazon and Google either couldn't or didn't bother to solve with their SimpleDB and Appengine Datastore offerings. Still sounds like quite a bold claim to me. But as said elsewhere, I appreciate that you're standing by it and will keep looking forward to what you will unveil. Just one more thing that I really don't understand: If you have this doomsday device at your hands, then why on earth do you bother with MySQL hosting as a secondary product?
- gfodor 17y ago
  You're misreading his argument or are being deliberately obtuse. The fact that you can scale the relational model using Oracle's software, with or without a support contract or appropriate tuning measures, is sufficient evidence to invalidate the claim that many make that the relational model does not scale. The relational model != ACID. The relational model != SQL. The relational model has nothing in it implicitly that ruins it's ability to obey CAP. You are right, that you will have to make some consistency tradeoffs to get partition tolerance. Again, to repeat, this has nothing to do with the relational model itself, just with the CAP theorem. Just because it is hard to build a RDBMS that loosens ACID but scales and is much easier to build a ad-hoc non-formally-defined model like that seen in Cassandra or BigTable does not mean it is impossible. You simply have to pay for it, in software, hardware, and consulting fees. Nothing is stopping the open source world from making PostgreSQL or MySQL scale out other than time, energy, and perhaps a seminal paper or two out of acadamic that provide foundational models that are necessary to implement it correctly. Odds are, they've done this work at Oracle, but it's not available for obvious reasons.
  - moe 17y ago
    You're misreading his argument or are being deliberately obtuse. Well, it seems you're misreading mine now. does not mean it is impossible. I never said that. If you re-read my post you will see that I was merely wondering about their claim of not just meeting the goalposts set by Oracle et al but of exceeding them by a large margin. As I understood it their new system is supposedly not only maintenance-free, but also efficient regardless of the workload or schema that I, as a user, might throw at it. And all that supposedly hosted on commodity VMs in a cloud (probably EC2). That's about as bold a claim as it gets in database-land, so if I was reading too much into it then feel free to correct me on that, but please don't suggest I said something that I never said. To put it into perspective (again): I would think both Google and Amazon have probably also thought long and hard about this problem. A full fledged SQL Store would be a much more sexy offering than their crippled SimpleDB/Datastore solutions after all. There's also a good amount of academic research in the area, with Cassandra (Dynamo/BigTable) seemingly being what many people consider the best we can currently do in terms of linear scalability on commodity hardware - at the cost of not having SQL and no ACID. So here comes this startup out of the blue, claiming to have solved all these things. Yes, leaps like that happen sometimes. Google happened, too, after all. But I think my skepticism is not completely unjustified either? Hence my question which tradeoff FathomDB chose to make, to get at least a basic idea about what to expect.
jbellis 17y ago
> you can pick up the phone and an Oracle salesperson will sell you the physical device that you're implying can't exist Bullshit, and I will elaborate. Oracle has two "scalable offerings:" RAC relies on a single large san to "scale." Exadata relies on super-fast interconnects to get multiple machines to look more like a single huge one, which has the obvious speed-of-light limitations as well as the price one. Neither approach is the kind of scale out with commodity hardware that something like Cassandra gives you. Invoking the mythical "pay Oracle enough money and they will make it scale" mantra is frequently done by people who either don't know better or are deliberately muddying the water, but that doesn't make it right.
- justinsb 17y ago
  Just because Oracle isn't using the exact same techniques as Cassandra, that doesn't make Oracle's ability to scale fictitious.
  - jbellis 17y ago
    I see you aced Straw Man 101. Did you not understand what I wrote, or do you think that "large san" or "fast interconnects limited to machines in the same couple of racks" count as "scaling?"
    - justinsb 17y ago
      Ouch - aren't you getting a little bit personal here? I'll try to remain professional, and I'll accept the fact that you're quoting things you didn't say, as the re-write was clearer as to your intention. When it comes to scaling, there's no rule which says you have to scale in any particular way. Many database customers consider scaling _up_ a form of scaling, and probably 99% of the world's database users will never go beyond what can be achieved on a single machine. They don't care about your pet project, about how the way it works is cooler and technologically purer. They consider databases a tool, and they don't really care how it works. I don't think too deeply about how a can opener works. I'm able to open more cans faster with a more powerful electric can opener. Even scaling up is still scaling. There are customers with bigger needs. For some of them, SAN based scaling is just what they need; more IOPS let their database keep running and they can get on with their lives. If they need more IOPS, they add more drives to their SAN. Scaling with a SAN is still scaling. Some customers have more complex demands, and look to Exadata or Netezza. They might start with just a few nodes, and add more nodes as their load increases. It's still scaling. Now, we in CompSci circles get excited about scaling using clusters. That's _sexy_ scaling. Cassandra and FathomDB can both change the rules of the game, and I'm excited about what FathomDB can do here, just as I can see you're passionate about Cassandra. But let's not pretend that most customers really care about how it scales; they care about what we can do. To customers, if it looks like a duck, and it quacks like a duck, it is a duck. But when they're choosing a database, scaling is not the only requirement, and certainly scaling in a certain way is unlikely to be on their list of requirements. If you've ever read an RFP, there's a bewildering number of questions that have nothing to do with technology at all, and often the technology section is a depressingly short list. Although part of the RFP game is to try to get the purchaser to write in requirements that only you can deliver, purchasers are considerably more savvy than that. You can jump up and down like Rumpelstiltskin arguing that yours is the only database that's _really_ scaling and the Oracle solution didn't meet the requirements, and that they should have chosen you. At the end of the day all you're left with is one person jumping up and down screaming about the rules of the game, and a customer that got what they needed and an Oracle salesperson that earned their commission.

moe 17y ago

but you can pick up the phone and an Oracle salesperson will sell you the physical device that you're implying can't exist. Well, thing is, part of your claim (if I understood it correctly) is that I just throw my data and my queries at your wonder-device and it will magically be fast and scale. You never mention any special care I must take, you even explicitly say it's not partitioning. This is decidedly not the case with what said Oracle salesperson will sell to me. The sale will normally include an expensive support-contract and probably a dedicated DB Admin. Because operating a RAC- or Exadata Cluster is, for all I know, far from trivial. You still have to make a bunch of critical decisions and tradeoffs upfront or it will probably suck. Moreover you're effectively saying that you solved what Amazon and Google either couldn't or didn't bother to solve with their SimpleDB and Appengine Datastore offerings. Still sounds like quite a bold claim to me. But as said elsewhere, I appreciate that you're standing by it and will keep looking forward to what you will unveil. Just one more thing that I really don't understand: If you have this doomsday device at your hands, then why on earth do you bother with MySQL hosting as a secondary product?

gfodor 17y ago
You're misreading his argument or are being deliberately obtuse. The fact that you can scale the relational model using Oracle's software, with or without a support contract or appropriate tuning measures, is sufficient evidence to invalidate the claim that many make that the relational model does not scale. The relational model != ACID. The relational model != SQL. The relational model has nothing in it implicitly that ruins it's ability to obey CAP. You are right, that you will have to make some consistency tradeoffs to get partition tolerance. Again, to repeat, this has nothing to do with the relational model itself, just with the CAP theorem. Just because it is hard to build a RDBMS that loosens ACID but scales and is much easier to build a ad-hoc non-formally-defined model like that seen in Cassandra or BigTable does not mean it is impossible. You simply have to pay for it, in software, hardware, and consulting fees. Nothing is stopping the open source world from making PostgreSQL or MySQL scale out other than time, energy, and perhaps a seminal paper or two out of acadamic that provide foundational models that are necessary to implement it correctly. Odds are, they've done this work at Oracle, but it's not available for obvious reasons.
- moe 17y ago
  You're misreading his argument or are being deliberately obtuse. Well, it seems you're misreading mine now. does not mean it is impossible. I never said that. If you re-read my post you will see that I was merely wondering about their claim of not just meeting the goalposts set by Oracle et al but of exceeding them by a large margin. As I understood it their new system is supposedly not only maintenance-free, but also efficient regardless of the workload or schema that I, as a user, might throw at it. And all that supposedly hosted on commodity VMs in a cloud (probably EC2). That's about as bold a claim as it gets in database-land, so if I was reading too much into it then feel free to correct me on that, but please don't suggest I said something that I never said. To put it into perspective (again): I would think both Google and Amazon have probably also thought long and hard about this problem. A full fledged SQL Store would be a much more sexy offering than their crippled SimpleDB/Datastore solutions after all. There's also a good amount of academic research in the area, with Cassandra (Dynamo/BigTable) seemingly being what many people consider the best we can currently do in terms of linear scalability on commodity hardware - at the cost of not having SQL and no ACID. So here comes this startup out of the blue, claiming to have solved all these things. Yes, leaps like that happen sometimes. Google happened, too, after all. But I think my skepticism is not completely unjustified either? Hence my question which tradeoff FathomDB chose to make, to get at least a basic idea about what to expect.

jbellis 17y ago

> you can pick up the phone and an Oracle salesperson will sell you the physical device that you're implying can't exist Bullshit, and I will elaborate. Oracle has two "scalable offerings:" RAC relies on a single large san to "scale." Exadata relies on super-fast interconnects to get multiple machines to look more like a single huge one, which has the obvious speed-of-light limitations as well as the price one. Neither approach is the kind of scale out with commodity hardware that something like Cassandra gives you. Invoking the mythical "pay Oracle enough money and they will make it scale" mantra is frequently done by people who either don't know better or are deliberately muddying the water, but that doesn't make it right.

justinsb 17y ago
Just because Oracle isn't using the exact same techniques as Cassandra, that doesn't make Oracle's ability to scale fictitious.
- jbellis 17y ago
  I see you aced Straw Man 101. Did you not understand what I wrote, or do you think that "large san" or "fast interconnects limited to machines in the same couple of racks" count as "scaling?"
  - justinsb 17y ago
    Ouch - aren't you getting a little bit personal here? I'll try to remain professional, and I'll accept the fact that you're quoting things you didn't say, as the re-write was clearer as to your intention. When it comes to scaling, there's no rule which says you have to scale in any particular way. Many database customers consider scaling _up_ a form of scaling, and probably 99% of the world's database users will never go beyond what can be achieved on a single machine. They don't care about your pet project, about how the way it works is cooler and technologically purer. They consider databases a tool, and they don't really care how it works. I don't think too deeply about how a can opener works. I'm able to open more cans faster with a more powerful electric can opener. Even scaling up is still scaling. There are customers with bigger needs. For some of them, SAN based scaling is just what they need; more IOPS let their database keep running and they can get on with their lives. If they need more IOPS, they add more drives to their SAN. Scaling with a SAN is still scaling. Some customers have more complex demands, and look to Exadata or Netezza. They might start with just a few nodes, and add more nodes as their load increases. It's still scaling. Now, we in CompSci circles get excited about scaling using clusters. That's _sexy_ scaling. Cassandra and FathomDB can both change the rules of the game, and I'm excited about what FathomDB can do here, just as I can see you're passionate about Cassandra. But let's not pretend that most customers really care about how it scales; they care about what we can do. To customers, if it looks like a duck, and it quacks like a duck, it is a duck. But when they're choosing a database, scaling is not the only requirement, and certainly scaling in a certain way is unlikely to be on their list of requirements. If you've ever read an RFP, there's a bewildering number of questions that have nothing to do with technology at all, and often the technology section is a depressingly short list. Although part of the RFP game is to try to get the purchaser to write in requirements that only you can deliver, purchasers are considerably more savvy than that. You can jump up and down like Rumpelstiltskin arguing that yours is the only database that's _really_ scaling and the Oracle solution didn't meet the requirements, and that they should have chosen you. At the end of the day all you're left with is one person jumping up and down screaming about the rules of the game, and a customer that got what they needed and an Oracle salesperson that earned their commission.