5 ms·

Interesting to see that they use two different db systems. I'm a bit worried about this. I believe it is just postponing an important decision. In the future th

Interesting to see that they use two different db systems. I'm a bit worried about this. I believe it is just postponing an important decision. In the future they will have to choose for one db system. With the growth of Twitter wouldn't it be better to make a decision now and focus all resources on optimizing one system?

gecko 16y ago
Meh. Many, many applications have at least two database systems: a caching layer, such as memcached; and a persistance layer, such as PostgreSQL. Twitter's basically just using Cassandra in place of memcached. This doesn't strike mas a dangerous or bizarre architecture.
- ora600 16y ago
  "Twitter's basically just using Cassandra in place of memcached" Not sure about that. They also said they are using it for analytics. Actually, since Cassandra is persistent to disks, using it as a memcached doesn't sound right.
  - babo 16y ago
    Since Cassandra 0.6 there is a way to keep data in memory, acting as a cache.
lox 16y ago
Why do they have to choose 1 database system? I never understand this reasoning. Pick the best tool for the job! I for one whole-heartedly agree with their decision. Considering the issues they've had recently with their internal networks, it would be insanity to move the majority of their stack to a new technology. MySQL scales just fine, provided you are good at partitioning and at least it's a known quantity.
deleted 16y ago
[deleted]
jackowayed 16y ago
There's no reason they have to use the same DB for everything; you use what's right for the task. Look at GitHub: they're running MySQL (or maybe PostgreSQL), Redis, and MongoDB. Facebook uses MySQL for most stuff, but also uses Hive and Casandra.
strlen 16y ago
One size doesn't fit all. MySQL and Cassandra are radically different system in both the query model and distribution models. Cassandra and HBase are similar in the data models (the differences are mandated by their respective distribution models), but are radically different in their distribution models. Entirely different systems are suited for entirely different applications.
- deleted 16y ago
  [deleted]
c3o 16y ago
They're also using their own FlockDB (http://github.com/twitter/flockdb http://github.com/twitter/flockdb) to store their social graph.
- alex1 16y ago
  FlockDB isn't a data store itself. They use FlockDB on top of MySQL AFAIK.
- xal 16y ago
  For actual data storage they build this layer on top of MYSQL ( and redis, and all other things that need sharding ) : http://github.com/twitter/gizzard http://github.com/twitter/gizzard
simonw 16y ago
To my mind, the whole NoSQL thing is primarily about "polyglot persistence" - building systems on more than one storage system. I think large scale apps that use more than one db system are becoming the norm. NoSQL means acknowledging that different storage systems are good for different features.
mbrubeck 16y ago
There's no reason a company can't use different tools for different jobs. When I worked at Amazon.com we used many storage and query systems: Oracle, MySQL, S3, BDB (and various simple distributed systems built on BDB), Dynamo, SimpleDB, memcached, etc. etc.

gecko 16y ago

Meh. Many, many applications have at least two database systems: a caching layer, such as memcached; and a persistance layer, such as PostgreSQL. Twitter's basically just using Cassandra in place of memcached. This doesn't strike mas a dangerous or bizarre architecture.

ora600 16y ago
"Twitter's basically just using Cassandra in place of memcached" Not sure about that. They also said they are using it for analytics. Actually, since Cassandra is persistent to disks, using it as a memcached doesn't sound right.
- babo 16y ago
  Since Cassandra 0.6 there is a way to keep data in memory, acting as a cache.

lox 16y ago

Why do they have to choose 1 database system? I never understand this reasoning. Pick the best tool for the job! I for one whole-heartedly agree with their decision. Considering the issues they've had recently with their internal networks, it would be insanity to move the majority of their stack to a new technology. MySQL scales just fine, provided you are good at partitioning and at least it's a known quantity.

deleted 16y ago

[deleted]

jackowayed 16y ago

There's no reason they have to use the same DB for everything; you use what's right for the task. Look at GitHub: they're running MySQL (or maybe PostgreSQL), Redis, and MongoDB. Facebook uses MySQL for most stuff, but also uses Hive and Casandra.

strlen 16y ago

One size doesn't fit all. MySQL and Cassandra are radically different system in both the query model and distribution models. Cassandra and HBase are similar in the data models (the differences are mandated by their respective distribution models), but are radically different in their distribution models. Entirely different systems are suited for entirely different applications.

deleted 16y ago
[deleted]

c3o 16y ago

They're also using their own FlockDB (http://github.com/twitter/flockdb http://github.com/twitter/flockdb) to store their social graph.

alex1 16y ago
FlockDB isn't a data store itself. They use FlockDB on top of MySQL AFAIK.
xal 16y ago
For actual data storage they build this layer on top of MYSQL ( and redis, and all other things that need sharding ) : http://github.com/twitter/gizzard http://github.com/twitter/gizzard

simonw 16y ago

To my mind, the whole NoSQL thing is primarily about "polyglot persistence" - building systems on more than one storage system. I think large scale apps that use more than one db system are becoming the norm. NoSQL means acknowledging that different storage systems are good for different features.

mbrubeck 16y ago

There's no reason a company can't use different tools for different jobs. When I worked at Amazon.com we used many storage and query systems: Oracle, MySQL, S3, BDB (and various simple distributed systems built on BDB), Dynamo, SimpleDB, memcached, etc. etc.