5 ms·
The "fanout" or "wasteful duplication of a single message 30 million times" is only required because they are using tiny underpowered hardware to begin with. T
by papsosouid 13y ago
The "fanout" or "wasteful duplication of a single message 30 million times" is only required because they are using tiny underpowered hardware to begin with. The approach that they claim can't possibly work actually does work. You just can't do it on a $2000 "server".
- marcofloriano 13y agoSo basically you are suggesting that it would be best for twitter to concentrate data and traffic into a couple servers. Let's say it's possible (better, lets imagine), so how would you handle a crash on a $200k server? Get two other $200k server backing it up? But what if your data and traffic needs not just ONE $200k server, but instead, a $200.000k server? You see that's not even close at how big twitter probably is.
- sulam 13y agoLet me guess, you work for Oracle?
- kingryan 13y agoThe messages are not repeated in fanout, just the ids. I guess you would know that if you actually the links.
- vidarh 13y agoThat still means 30 million writes instead of one. That the size of each write is smaller is not going to help you all that much if each write still worst case ends up forcing you to rewrite at least one disk sector. While I'm inclined to favour a fanout approach, a "full" fanout can easily be incredibly costly on the write side.