4 ms·

I don't know about you, but to me this seems like a big blow to the reputation of Cassandra. I know a bunch of people (including myself) whose primary motivatio

by seldo 16y ago

I don't know about you, but to me this seems like a big blow to the reputation of Cassandra. I know a bunch of people (including myself) whose primary motivation for investigating Cassandra as a large-scale data store was "Twitter is using it, therefore it's got to be fine for what we want". In my experience this was not the case -- after quite a lot of prototyping, we moved away from Cassandra. It scales brilliantly, but its query time has a pretty solid floor that I was unable to shift to speeds approaching what I needed: it never went up, but nor did it go down. Of course, I am a total noob at distributed systems -- perhaps it's simple to get it to go faster, but Twitter's move away suggests not. (Facebook's use-case is quite different, and in any case they still use an awful lot of MySQL too) The other systems they cite as currently using Cassandra in production are nowhere near the scale of the primary tweet store.

alex1 16y ago
I don't think they're avoiding Cassandra as their tweet store because it's not capable of performing well. It's probably more cause it would be a really long process migrating every tweet into Cassandra. I'm sure if they could, they would go back to the first day Twitter launched, and use Cassandra as the tweets store. As for query time, maybe your problem was I/O throughput? I've read Cassandra can be really slow if you have poor I/O throughput. This happens a lot in the cloud, where I/O is usually shared.
- jacobbijani 16y ago
  Don't they not show / hide & archive old tweets anyway? Seems like they could really simple transition Cassandra in if they wanted to.
  - mrduncan 16y ago
    On a small scale, yes. On Twitter scale (in February they were averaging 50 million tweets/day [1]), it's probably not so simple. [1] http://blog.twitter.com/2010/02/measuring-tweets.html http://blog.twitter.com/2010/02/measuring-tweets.html Edit: As of mid June, it was about 65 million per day. http://blog.twitter.com/2010/06/big-goals-big-game-big-records.html http://blog.twitter.com/2010/06/big-goals-big-game-big-recor...
    - jacobbijani 16y ago
      My point was that they aren't importing legacy data at all if they aren't showing old stuff. Just start writing the new ones to the new DB, right?

by seldo 16y ago

alex1 16y ago

I don't think they're avoiding Cassandra as their tweet store because it's not capable of performing well. It's probably more cause it would be a really long process migrating every tweet into Cassandra. I'm sure if they could, they would go back to the first day Twitter launched, and use Cassandra as the tweets store. As for query time, maybe your problem was I/O throughput? I've read Cassandra can be really slow if you have poor I/O throughput. This happens a lot in the cloud, where I/O is usually shared.

jacobbijani 16y ago
Don't they not show / hide & archive old tweets anyway? Seems like they could really simple transition Cassandra in if they wanted to.
- mrduncan 16y ago
  On a small scale, yes. On Twitter scale (in February they were averaging 50 million tweets/day [1]), it's probably not so simple. [1] http://blog.twitter.com/2010/02/measuring-tweets.html http://blog.twitter.com/2010/02/measuring-tweets.html Edit: As of mid June, it was about 65 million per day. http://blog.twitter.com/2010/06/big-goals-big-game-big-records.html http://blog.twitter.com/2010/06/big-goals-big-game-big-recor...
  - jacobbijani 16y ago
    My point was that they aren't importing legacy data at all if they aren't showing old stuff. Just start writing the new ones to the new DB, right?