5 ms·

> We currently use one database for writes with multiple slaves for read queries. As many know, replication of MySQL is no easy task, so we've brought in MySQL

> We currently use one database for writes with multiple slaves for read queries. As many know, replication of MySQL is no easy task, so we've brought in MySQL experts to help us with that immediately. This really makes me wonder if they know what they're doing. MySQL replication really isn't that hard at all. They're hitting performance bottlenecks with only 3 servers and don't appear to be looking into sharding/partitioning to split up writes?

briansmith 18y ago
It looks to me like he is saying they are planning on ditching MySQL altogether. That is probably why they don't want to invest in redoing/expanding the MySQL replication.
- chrisbolt 18y ago
  Twitter isn't that complicated of a website, it shouldn't take them more than a few days to migrate only the high load sections (tweets) to a sharded architecture. All they need is money for more database servers (and they have at least $15 million) and know-how. And yet they're planning to ditch MySQL in favour of... text files? And they can't even make a small MySQL cluster highly available?
  - tonystubblebine 18y ago
    Here's an interesting proposed architecture that involves ditching MySQL: http://randomfoo.net/blog/id/4182 http://randomfoo.net/blog/id/4182
swombat 18y ago
On the contrary, I'd say that they must really know what they're doing. How many sites with a million daily active users do you know that run on only 3 db servers? The twitter team deserves a lot of respect for managing to stretch their hardware so far.
- gaius 18y ago
  smirk The company I work for serves volumes that dwarf that of Twitter. We don't drop anything, because we are handling actual money, not tweets. And we get by with 3 database servers, one of which is for failover. OK admittedly they are wardrobe-sized Suns running Oracle, but that's not the point. Yes it takes skill to do a lot with a little, but it takes skill not to paint yourself into a corner too, and you need both skills to succeed.
  - swombat 18y ago
    I'd point out that 3 wardrobe-size Sun servers running Oracle probably cost about as much as 300 MySQL servers like Tweeter is running, but some might say I'm the one being a smartass then :-P
- jonknee 18y ago
  Twitter definitely does not have millions of daily active users. There are only 824M total tweets. Comparing IDs from posts 24 hours apart shows there were ~500,000 tweets in the past 24 hours.
  - swombat 18y ago
    I didn't say "millions", I said "a million". That's according to their own reports, and if there's 500'000 tweets, it's quite likely there's a million people sitting there with a tweeter client open (not everyone tweets every day). Daniel
    - jonknee 18y ago
      Your guess is still way high. It's around 200,000 active per week. Probably close to that every day as well since it's a dedicated group. http://www.techcrunch.com/2008/04/29/end-of-speculation-the-real-twitter-usage-numbers/ http://www.techcrunch.com/2008/04/29/end-of-speculation-the-...

briansmith 18y ago

It looks to me like he is saying they are planning on ditching MySQL altogether. That is probably why they don't want to invest in redoing/expanding the MySQL replication.

chrisbolt 18y ago
Twitter isn't that complicated of a website, it shouldn't take them more than a few days to migrate only the high load sections (tweets) to a sharded architecture. All they need is money for more database servers (and they have at least $15 million) and know-how. And yet they're planning to ditch MySQL in favour of... text files? And they can't even make a small MySQL cluster highly available?
- tonystubblebine 18y ago
  Here's an interesting proposed architecture that involves ditching MySQL: http://randomfoo.net/blog/id/4182 http://randomfoo.net/blog/id/4182

swombat 18y ago

On the contrary, I'd say that they must really know what they're doing. How many sites with a million daily active users do you know that run on only 3 db servers? The twitter team deserves a lot of respect for managing to stretch their hardware so far.

gaius 18y ago
smirk The company I work for serves volumes that dwarf that of Twitter. We don't drop anything, because we are handling actual money, not tweets. And we get by with 3 database servers, one of which is for failover. OK admittedly they are wardrobe-sized Suns running Oracle, but that's not the point. Yes it takes skill to do a lot with a little, but it takes skill not to paint yourself into a corner too, and you need both skills to succeed.
- swombat 18y ago
  I'd point out that 3 wardrobe-size Sun servers running Oracle probably cost about as much as 300 MySQL servers like Tweeter is running, but some might say I'm the one being a smartass then :-P
jonknee 18y ago
Twitter definitely does not have millions of daily active users. There are only 824M total tweets. Comparing IDs from posts 24 hours apart shows there were ~500,000 tweets in the past 24 hours.
- swombat 18y ago
  I didn't say "millions", I said "a million". That's according to their own reports, and if there's 500'000 tweets, it's quite likely there's a million people sitting there with a tweeter client open (not everyone tweets every day). Daniel
  - jonknee 18y ago
    Your guess is still way high. It's around 200,000 active per week. Probably close to that every day as well since it's a dedicated group. http://www.techcrunch.com/2008/04/29/end-of-speculation-the-real-twitter-usage-numbers/ http://www.techcrunch.com/2008/04/29/end-of-speculation-the-...