2 ms·
You get diminishing returns, of course, on any kind of technology. You would just basically partition this data out to a 1000 different clusters then, each capa
by kenshaw 12y ago
You get diminishing returns, of course, on any kind of technology. You would just basically partition this data out to a 1000 different clusters then, each capable of supporting 1 million writes/second. That would be done through an algorithm along the lines of this:
partition_id = well_distributed_hash(normalize_domain_name(<domain>))
really don't think this would be an issue, or even a concern, as 1 billion writes per second are "Google-levels" of writes. One would likely run out of financial resources supporting an expansion of 86 terrabytes / day (assuming 1kb minimum new data per write), far faster than you would ever max out the capabilities of the technology.
It's more a finance/business exercise to determine if revenue inflows would actually meet, cover, or exceed revenue outflows on a project like this. You can always make the technology scale and meet the demand.