3 ms·
More info at https://technology.blog.gov.uk/2016/08/16/scaling-the-petitions-service-following-the-eu-referendum/ https://technology.blog.gov.uk/2016/08/16/scal
by pbhjpbhj 8y ago
More info at https://technology.blog.gov.uk/2016/08/16/scaling-the-petitions-service-following-the-eu-referendum/ https://technology.blog.gov.uk/2016/08/16/scaling-the-petiti....
I'm not a db person, but are they sharding. Surely as they're only writing simple tuples with a petition they can easily write to multiple separate dbs (shards I guess) and amalgamate the results at leisure?
That link says they need to update the web content perpetually, but it seems it's only the number of signatories they need to keep live, isn't it. So once it breeches 100k then they can estimate the signatory rate (including acceleration) and update the row count when they think 200k will be breeched - at 1k/min updating the row count each hour is fine, presumably. Surely that's not a massive processing job?
I'd love some input here, because naively it seems like they're not handling that much data? A million rows of 1kB is still only 1GB, and they're only gathering a few simple fields in each tuple??