Like others have pointed out, justin.tv's main issue in cases like this would be serving the same data to lots of users, aka bandwidth, which can pretty easily

Like others have pointed out, justin.tv's main issue in cases like this would be serving the same data to lots of users, aka bandwidth, which can pretty easily solved. Twitter however has to fetch random data for million of different users, and hence simply throwing more bandwidth at it wont solve anything at all. It basically boils down to making a good and sane database-design which allows rapid updates and lookups for the core uses, and minimizing the load caused by non-core uses. That and using a database engine which doesn't fall apart when the dataset gets bigger than whatever amounts of RAM you can throw into your server. Evidently twitter could have made better choices on both those accounts.