5 ms·

"While in theory, NoSQL databases can be deployed on hundreds or thousands of machines, in practice the deployments are much smaller and can typically easily fi

by nphase 15y ago

"While in theory, NoSQL databases can be deployed on hundreds or thousands of machines, in practice the deployments are much smaller and can typically easily fit within a single Oracle database with fewer and more reliable components. The largest production Cassandra cluster has over 100 TB of data in over 150 machines." I find this interesting. I wonder if there any real TCO comparisons have been done to this point. Sure, this paper is full of FUD. But it does raise a valid question for me: What's the difference between paying for Oracle and paying for MySQL, Datastax, or Cloudera support/enterprise to get the added features needed to run my business and keep away from reinventing the wheel?

shin_lao 15y ago
That sounds bogus. A 100 TiB Oracle database is doable but incredibly expensive. 1 TiB with the appropriate redundancy will cost you around 10,000 € if you go Oracle. Perhaps you can get a good discount if you order a large cluster right away. So we're looking at 1,000,000 € just for disks here. Then you would have to pay for the CPUs, the licenses and the consulting (lots of it, tuning a 100 TiB system is a huge task). Add another million or two... How much would it cost with a NoSQL solution? Don't want to turn this answer into self promotion, but I'm confident it's well under the million with better performance (assuming you don't need a relational database).
- Tractiable 15y ago
  How can you self promote something you can't confidently describe?
- sliverstorm 15y ago
  They probably give you volume discounts, you know. And would a 150 machine NoSQL cluster really compare so favorably? An enterprise-grade server with an array of 10k-15k RPM drives or SSD's and appropriate redundancy is not going to be cheap. What about the cost of manpower to operate one vs. the other? If you can hire one novice sysadmin @ $40k, labor costs will be small, but for a 150-machine cluster you aren't going to get far with a $40k novice.
  - shin_lao 15y ago
    The price I gave you is what one of our customers pay, after negotiation. True, they didn't order for 100 TiB. In 2011, you don't need 150 nodes to build a 100 TiB system. Don't take this as an official estimate, but I think we would advise on 7200 drives and more RAM and it would probably end up with a system one (several?) order(s) of magnitude faster than what you would get with a regular RDBMS.
    - nknight 15y ago
      Hell, you could probably fit 100TB in pure RAM for what you'd spend on Oracle.
      - mitchty 15y ago
        Depends on the hardware and how much ram you want in a system: http://www.sgi.com/products/servers/altix/uv/configs.html http://www.sgi.com/products/servers/altix/uv/configs.html Gives you up to 16Tb of ram in one system, but you'll pay. We just got two, they are insane beasts but cool.
      - sliverstorm 15y ago
        Good luck with that. At $100 per 8GB of ECC RAM, that's $1mil in RAM, and with 4-slot machines you'll need 3,200 boxes. 1,600 boxes if you go for dual-CPU machines with 8-slot.
        onemoreact 15y ago
        You can get dedicated machines that hold 16TB of ram in 18U. Also you can get 2x8GB of ECC ram for 150$ and with bulk orders it drops even more.
        nknight 15y ago
        First, 8GB ECC sticks are actually a bit under $100... retail. You're not going to buy almost 13,000 sticks of RAM retail, are you? Second, server motherboards are not limited to a mere 8 slots. 16's are widely available, even 32's can be found at retail, and there are more options out there than what Newegg or whatever you're shopping at has.
- hapless 15y ago
  I have never met anyone who pays Oracle's list prices. It sure seems like every Oracle deal is a one-off.
rbanffy 15y ago
Truth is, most of the Oracle (same applies do SQL Server) databases I've met could run just fine under MySQL or PostgreSQL. A good deal of them would be unable to significantly stress SQLite or MS Access. Unless you start with a million users generating data, NoSQL (or Oracle RDBMS) is premature optimization. You want to architect for growth just in case, fine, but don't waste your time before you know you will have a problem.
- TylerE 15y ago
  I see this sentiment a lot. It annoys me. NoSQL is not a performance operation. It's about choosing a representation that is appropriate for your data.
  - rbanffy 15y ago
    > NoSQL is not a performance operation. It becomes one, but only when your dataset reaches sizes almost nobody will ever see. If your business model implies a scale-or-die condition, then I'd say it's fair to design your solution to accommodate NoSQL, but not waste time implementing it in production. It's a horrible waste of time to design something around a Cassandra cluster that will be hit once every second. And, BTW, if you hit the performance architecture's limit (actually, you should have detailed monitoring to tell you how many weeks you have until you hit it), then you should consider what you'll do to solve your particular problem (which may not be what you anticipated initially). You should always invest a little time evaluating your options for different scenarios.
    - randomdata 15y ago
      It's a horrible waste of time to design something around a Cassandra cluster that will be hit once every second. While I would probably agree that Cassandra was designed for scalability, there are hundreds of other NoSQL databases designed for completely different problems that have nothing to do with scaling. CouchDB, for instance, is really good at synchronizing data between separate disjoint databases (think temporary offline use); a problem some have with databases both big and small. Every NoSQL database was born out of the need to solve a problem that was not well solved using existing technologies. Some of those problems are related to scaling, but scaling problems are just the cusp of what is now available to you to use. Definitely use SQL when it is the right tool for the job, but don't try to shoehorn it into the wrong job. There are other tools that just might be a better fit no matter how big your database is or how many people are going to be using it.
deleted 15y ago
[deleted]

by nphase 15y ago

shin_lao 15y ago

That sounds bogus. A 100 TiB Oracle database is doable but incredibly expensive. 1 TiB with the appropriate redundancy will cost you around 10,000 € if you go Oracle. Perhaps you can get a good discount if you order a large cluster right away. So we're looking at 1,000,000 € just for disks here. Then you would have to pay for the CPUs, the licenses and the consulting (lots of it, tuning a 100 TiB system is a huge task). Add another million or two... How much would it cost with a NoSQL solution? Don't want to turn this answer into self promotion, but I'm confident it's well under the million with better performance (assuming you don't need a relational database).

Tractiable 15y ago
How can you self promote something you can't confidently describe?
sliverstorm 15y ago
They probably give you volume discounts, you know. And would a 150 machine NoSQL cluster really compare so favorably? An enterprise-grade server with an array of 10k-15k RPM drives or SSD's and appropriate redundancy is not going to be cheap. What about the cost of manpower to operate one vs. the other? If you can hire one novice sysadmin @ $40k, labor costs will be small, but for a 150-machine cluster you aren't going to get far with a $40k novice.
- shin_lao 15y ago
  The price I gave you is what one of our customers pay, after negotiation. True, they didn't order for 100 TiB. In 2011, you don't need 150 nodes to build a 100 TiB system. Don't take this as an official estimate, but I think we would advise on 7200 drives and more RAM and it would probably end up with a system one (several?) order(s) of magnitude faster than what you would get with a regular RDBMS.
  - nknight 15y ago
    Hell, you could probably fit 100TB in pure RAM for what you'd spend on Oracle.
    - mitchty 15y ago
      Depends on the hardware and how much ram you want in a system: http://www.sgi.com/products/servers/altix/uv/configs.html http://www.sgi.com/products/servers/altix/uv/configs.html Gives you up to 16Tb of ram in one system, but you'll pay. We just got two, they are insane beasts but cool.
    - sliverstorm 15y ago
      Good luck with that. At $100 per 8GB of ECC RAM, that's $1mil in RAM, and with 4-slot machines you'll need 3,200 boxes. 1,600 boxes if you go for dual-CPU machines with 8-slot.
      - onemoreact 15y ago
        You can get dedicated machines that hold 16TB of ram in 18U. Also you can get 2x8GB of ECC ram for 150$ and with bulk orders it drops even more.
      - nknight 15y ago
        First, 8GB ECC sticks are actually a bit under $100... retail. You're not going to buy almost 13,000 sticks of RAM retail, are you? Second, server motherboards are not limited to a mere 8 slots. 16's are widely available, even 32's can be found at retail, and there are more options out there than what Newegg or whatever you're shopping at has.
hapless 15y ago
I have never met anyone who pays Oracle's list prices. It sure seems like every Oracle deal is a one-off.

rbanffy 15y ago

Truth is, most of the Oracle (same applies do SQL Server) databases I've met could run just fine under MySQL or PostgreSQL. A good deal of them would be unable to significantly stress SQLite or MS Access. Unless you start with a million users generating data, NoSQL (or Oracle RDBMS) is premature optimization. You want to architect for growth just in case, fine, but don't waste your time before you know you will have a problem.

TylerE 15y ago
I see this sentiment a lot. It annoys me. NoSQL is not a performance operation. It's about choosing a representation that is appropriate for your data.
- rbanffy 15y ago
  > NoSQL is not a performance operation. It becomes one, but only when your dataset reaches sizes almost nobody will ever see. If your business model implies a scale-or-die condition, then I'd say it's fair to design your solution to accommodate NoSQL, but not waste time implementing it in production. It's a horrible waste of time to design something around a Cassandra cluster that will be hit once every second. And, BTW, if you hit the performance architecture's limit (actually, you should have detailed monitoring to tell you how many weeks you have until you hit it), then you should consider what you'll do to solve your particular problem (which may not be what you anticipated initially). You should always invest a little time evaluating your options for different scenarios.
  - randomdata 15y ago
    It's a horrible waste of time to design something around a Cassandra cluster that will be hit once every second. While I would probably agree that Cassandra was designed for scalability, there are hundreds of other NoSQL databases designed for completely different problems that have nothing to do with scaling. CouchDB, for instance, is really good at synchronizing data between separate disjoint databases (think temporary offline use); a problem some have with databases both big and small. Every NoSQL database was born out of the need to solve a problem that was not well solved using existing technologies. Some of those problems are related to scaling, but scaling problems are just the cusp of what is now available to you to use. Definitely use SQL when it is the right tool for the job, but don't try to shoehorn it into the wrong job. There are other tools that just might be a better fit no matter how big your database is or how many people are going to be using it.

deleted 15y ago

[deleted]