3 ms·
It's not like Twitter can't offer some slower Firehose or more advanced filtering or whatever. Or offer some EC2 images to spin up to handle it. It's only 500M
by MichaelGG 12y ago
It's not like Twitter can't offer some slower Firehose or more advanced filtering or whatever. Or offer some EC2 images to spin up to handle it.
It's only 500M items a day. At that level you've got plenty of options as far as hardware and software. (You could even record it into a normalass SQL DB to log each one if you wanted.) You may need some useful buffering for high spikes, perhaps some message queues with lots of RAM.
- ultimape 12y agoThis made me giggle
- MichaelGG 12y agoWhy? A couple years ago I was running a telecom, and we had around that volume of calls a day. Each call required not only routing instructions that hit several DBs, but also a transactional INSERT/UPDATE to account balances. Single node SQL Server handled it fine. (Though it was less than optimal at doing non-OLTP stuff.) Twitter can handle this for people (like DataSift, providing historical lookback), or they can provide template to help them spin up. It's not gonna cost thousands of dollars a month in hardware.