4 ms·

"1 MySQL Server (one big 8 core box) and 1 slave. Slave is read only for statistics and reporting." is twitter really running on only one mysql server, or is t

by growt 17y ago

"1 MySQL Server (one big 8 core box) and 1 slave. Slave is read only for statistics and reporting." is twitter really running on only one mysql server, or is this information outdated? no sharding, nothing?

mahmud 17y ago
This particular design seems to be the official one, but twitter and its lackluster architecture has spawned an industry of twitter optimizers. Everyone and their brother is trying to do something about the fail-whale: I mean, "how hard could it be"? the test implementations are usually benchmarked off of 127.0.0.1 and the "lab" results are, theoretically, impressive. A similar thing happened when reddit switched to python from common lisp; a flurry of reddit clones in Lisp, each smaller/faster/"eleganter" (pick two) than the other.
mdasen 17y ago
You'd be surprised how much one can run off one server should the application not need a lot of reads. Every single user page should be able to be cached until the user posts again. Likewise, the page of any post should be able to be cached infinitely. They don't run their messaging queue off their database I'd assume. Really, Twitter doesn't do relational data which makes scaling a good bit easier. The small amount of relational data that they do could be easily denormalized and pushed into memcached. The problem that a lot of web apps face is that it's hard to denormalize because so much of the data is relational. The only relations that I can think of is Person follows Person and Post belongs to Person and Post replies to Person. So, it's not hard to denormalize that in a way that you can push into memcached rather than having to hit the database much. I'm not saying that's how I'd do it if I were building Twitter, but with minimal relations and a small amount of data it's easy enough to keep the MySQL store in memory for log(n) lookups from memory and use memcached for O(1) lookups for the majority of stuff.
fr0z3nph03n1x 17y ago
I think this is outdated. It says 250k users. If aplusk has over a million followers this number can not be correct.
- jmonegro 17y ago
  This is a pre-Oprah article.

by growt 17y ago

mahmud 17y ago

This particular design seems to be the official one, but twitter and its lackluster architecture has spawned an industry of twitter optimizers. Everyone and their brother is trying to do something about the fail-whale: I mean, "how hard could it be"? the test implementations are usually benchmarked off of 127.0.0.1 and the "lab" results are, theoretically, impressive. A similar thing happened when reddit switched to python from common lisp; a flurry of reddit clones in Lisp, each smaller/faster/"eleganter" (pick two) than the other.

mdasen 17y ago

You'd be surprised how much one can run off one server should the application not need a lot of reads. Every single user page should be able to be cached until the user posts again. Likewise, the page of any post should be able to be cached infinitely. They don't run their messaging queue off their database I'd assume. Really, Twitter doesn't do relational data which makes scaling a good bit easier. The small amount of relational data that they do could be easily denormalized and pushed into memcached. The problem that a lot of web apps face is that it's hard to denormalize because so much of the data is relational. The only relations that I can think of is Person follows Person and Post belongs to Person and Post replies to Person. So, it's not hard to denormalize that in a way that you can push into memcached rather than having to hit the database much. I'm not saying that's how I'd do it if I were building Twitter, but with minimal relations and a small amount of data it's easy enough to keep the MySQL store in memory for log(n) lookups from memory and use memcached for O(1) lookups for the majority of stuff.

fr0z3nph03n1x 17y ago

I think this is outdated. It says 250k users. If aplusk has over a million followers this number can not be correct.

jmonegro 17y ago
This is a pre-Oprah article.