4 ms·

Not all businesses would go out of business if some data was not backed up. If Chartbeat lost its data, it would be a minor inconvenience that the feature of th

Not all businesses would go out of business if some data was not backed up. If Chartbeat lost its data, it would be a minor inconvenience that the feature of their real-time dashboard where you can rewind and look back at an earlier point of the day wouldn't work until it repopulated the next day. They probably wouldn't even lose a customer over it. When they were a young company without much funding, it might've made business sense to only back up their accounts and not the historical data for the sites being tracked. Not all data can be backed up for only $1000 a year either; it's not just a matter of storage costs and a server. A write-heavy service like Foursquare can't just turn off and do a dump at night; to have backups they need enough server capacity to replicate data as it comes in. For those servers to keep up, they have to be as beefy as the ones they're replicating -- a year or so ago, that was 64GB RAM per database for their Mongo instances. That's definitely more than $1000 a year's worth.

beagle3 15y ago
> a year or so ago, that was 64GB RAM per database for their Mongo instances. That's definitely more than $1000 a year's worth. You are mixing data, memory and processing power. In the case Amazon crashes and burns, they need that CPU+Ram+bandwidth etc. near the data. But to just keep a copy of the data (not available online) they don't need a really beefy server -- just store all the updates from a node, and replay them later at your leisure. 5400rpm will often be fast enough for this purpose (of course, you would need days to recover in this setup ... but that might be ok if past data is not needed online) Take twitter or foursquare for example: They need everyone's most recent max(4, tweets/checkins in the last day) available online at any given time. But if last years' tweets/checkins are not available online when the system is degraded in case of backup restoration etc -- that may be acceptable.
- deleted 15y ago
  [deleted]

beagle3 15y ago

> a year or so ago, that was 64GB RAM per database for their Mongo instances. That's definitely more than $1000 a year's worth. You are mixing data, memory and processing power. In the case Amazon crashes and burns, they need that CPU+Ram+bandwidth etc. near the data. But to just keep a copy of the data (not available online) they don't need a really beefy server -- just store all the updates from a node, and replay them later at your leisure. 5400rpm will often be fast enough for this purpose (of course, you would need days to recover in this setup ... but that might be ok if past data is not needed online) Take twitter or foursquare for example: They need everyone's most recent max(4, tweets/checkins in the last day) available online at any given time. But if last years' tweets/checkins are not available online when the system is degraded in case of backup restoration etc -- that may be acceptable.

deleted 15y ago
[deleted]