4 ms·

Wow: > For this feature, the fully denormalized Cassandra dataset weighs in at 3 terabytes and 76 billion columns.

by mcav 17y ago

breck 17y ago
What's amazing is that 3 terabyes costs about $300 nowadays. Include good components, power, redundancy, maintenance, and it probably comes to $1,000 per year. So this feature is actually pretty cheap to do, unless I'm missing something.
- SwellJoe 17y ago
  Every time I read "terabyte", I have to pause, because my brain goes "That's gigantic!" and then a little voice says, "It's the same size as the hard disk in my desktop...it cost $89." I have the same sort of weird mental disconnect with SD cards. It's just hard to fathom 32GB of data fitting into something about the size of a quarter.
  - imajes 17y ago
    also note that WD (?) have managed to make a terabyte 2.5" disk. Expect high-end laptops to ship with that soon enough. :)
  - willy1234x1 17y ago
    Hey, I know the feeling, it's weird to adjust to but I still haven't filled my 5 Terabytes of space yet so I'm happy. Soon I'm going to offload all my files to a file-server I'm building with an exorbitant amount of HDD space.
- jbellis 17y ago
  Serving up thousands of requests per seconds from 3 TB (while continually adding and updating more data!) is a lot different from just archiving it. A _lot_. :) To put it in perspective, 100GB is pretty much the limit of what you can serve from a single mysql machine, for instance. This will vary depending on exact workload but that is a reasonable ball park number.
  - SwellJoe 17y ago
    Of course, in a relational database you wouldn't (generally) be storing the vast majority of the data in question...you'd be querying for it. That's not to trivialize the improvement in performance here, just to point out that this probably would be 100GB in a MySQL database.
  - ntoshev 17y ago
    Any idea how much per machine can you serve when you use a bigtable/cassandra/dynamo/pnuts-like system instead of MySQL?
    - ntoshev 17y ago
      http://wiki.apache.org/cassandra/PoweredBy http://wiki.apache.org/cassandra/PoweredBy 330GB per machine for Facebook's Cassandra cluster.
- joechung 17y ago
  3 TB of disk is easy. 3 TB flowing quickly from disk to RAM to CPU cache and back out again is another story.
  - rbanffy 17y ago
    I am most of the time disappointed by x86s, but i5/i7 looks more promising: http://arstechnica.com/hardware/news/2009/09/intel-launches-all-new-pc-architecture-with-core-i5i7-cpus.ars http://arstechnica.com/hardware/news/2009/09/intel-launches-...
pgebhard 17y ago
I was thinking "WOW", as well, but breck's right. This shouldn't really be that surprising any more given the price of disk storage these days.

Wow: > For this feature, the fully denormalized Cassandra dataset weighs in at 3 terabytes and 76 billion columns.

by mcav 17y ago

Wow: > For this feature, the fully denormalized Cassandra dataset weighs in at 3 terabytes and 76 billion columns.

breck 17y ago

What's amazing is that 3 terabyes costs about $300 nowadays. Include good components, power, redundancy, maintenance, and it probably comes to $1,000 per year. So this feature is actually pretty cheap to do, unless I'm missing something.

SwellJoe 17y ago
Every time I read "terabyte", I have to pause, because my brain goes "That's gigantic!" and then a little voice says, "It's the same size as the hard disk in my desktop...it cost $89." I have the same sort of weird mental disconnect with SD cards. It's just hard to fathom 32GB of data fitting into something about the size of a quarter.
- imajes 17y ago
  also note that WD (?) have managed to make a terabyte 2.5" disk. Expect high-end laptops to ship with that soon enough. :)
- willy1234x1 17y ago
  Hey, I know the feeling, it's weird to adjust to but I still haven't filled my 5 Terabytes of space yet so I'm happy. Soon I'm going to offload all my files to a file-server I'm building with an exorbitant amount of HDD space.
jbellis 17y ago
Serving up thousands of requests per seconds from 3 TB (while continually adding and updating more data!) is a lot different from just archiving it. A _lot_. :) To put it in perspective, 100GB is pretty much the limit of what you can serve from a single mysql machine, for instance. This will vary depending on exact workload but that is a reasonable ball park number.
- SwellJoe 17y ago
  Of course, in a relational database you wouldn't (generally) be storing the vast majority of the data in question...you'd be querying for it. That's not to trivialize the improvement in performance here, just to point out that this probably would be 100GB in a MySQL database.
- ntoshev 17y ago
  Any idea how much per machine can you serve when you use a bigtable/cassandra/dynamo/pnuts-like system instead of MySQL?
  - ntoshev 17y ago
    http://wiki.apache.org/cassandra/PoweredBy http://wiki.apache.org/cassandra/PoweredBy 330GB per machine for Facebook's Cassandra cluster.
joechung 17y ago
3 TB of disk is easy. 3 TB flowing quickly from disk to RAM to CPU cache and back out again is another story.
- rbanffy 17y ago
  I am most of the time disappointed by x86s, but i5/i7 looks more promising: http://arstechnica.com/hardware/news/2009/09/intel-launches-all-new-pc-architecture-with-core-i5i7-cpus.ars http://arstechnica.com/hardware/news/2009/09/intel-launches-...

pgebhard 17y ago

I was thinking "WOW", as well, but breck's right. This shouldn't really be that surprising any more given the price of disk storage these days.