4 ms·
This. Outside of FAANG, Gov and finance, very few consumer facing companies have the budgets to engineer redundancies.
by ChuckNorris89 6y ago
This. Outside of FAANG, Gov and finance, very few consumer facing companies have the budgets to engineer redundancies.
- nix23 6y ago>budgets to engineer redundancies You dont need that, but a backup ;)
- virgilp 6y agoThey obviously do have _some_ backups (since were able to restore all videos and all photos - but not all photos at the original resolution). In the end it's all a tradeoff... ignoring cost, we all know what to do to get pretty safe storage. But how much are you really willing to pay, per month, for that safety?
- nix23 6y ago>but not all photos at the original resolution Thats why you backup the original data and NOT the downscaled etc one. >But how much are you really willing to pay, per month, for that safety? You have to backup ALL your customers data, otherwise DONT go into file-hosting business.
- ChuckNorris89 6y agoWith your argument, Google photos should also not be in business since they also don't back up your original resolution unless you pay.
- virgilp 6y agoBut they did backup the original data - the article states that the problem was with deploying wrong cleanup code on the long-term storage (that was supposed to be only deployed for the short-term storage). How do you guard against that? You could store everything in backup storage & never delete it, but that has implications: - privacy - legal (GDPR) - cost etc. It's basically a "black swan" event, they designed their system without anticipating this particular failure mode. It's easy to claim in hindsight that "they should've or they should not go into that business" but seriously.... look in the mirror. Are you as tough on yourself when it comes to your customers? Can you honestly claim that there's absolutely no failure mode that could result in customer data loss?
- nix23 6y ago> privacy Encryption? > legal (GDPR) You dont have to delete offline backups...never heard of such a case > cost etc. Cheaper than loose your customer's data wich they trust you with. EDIT: If you can delete a backup it's not a backup.
- virgilp 6y agoUnder GDPR "right to be forgotten", you have to delete backups, can't keep them forever - see https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection-regulation-gdpr/individual-rights/right-to-erasure/ https://ico.org.uk/for-organisations/guide-to-data-protectio... There are alternatives (one of which is to encrypt the backed up data & just throw away the encryption keys instead) but of course those incur additional costs > Cheaper than loose your customer's data wich they trust you with. This is just opinion. And also it's heavily context-dependent(cost of losing customer data is not the same for all data or all businesses). > EDIT: If you can delete a backup it's not a backup. If it gets destroyed, is it a backup? How's that different from "delete"? Those were not intentionally deleted, they were accidentally destroyed. Can your backups be accidentally destroyed?
- nix23 6y ago>you have to delete backups, can't keep them forever >>The key issue is to put the backup data ‘beyond use’, even if it cannot be immediately overwritten. It's clear that you cannot delete WORM-tapen with many other data on it. >This is just opinion. Not for a file hosting service, the customer trusts you to do your job. >If it gets destroyed, is it a backup? Destroyed means physikal interaction, since it is not possible to make a reallife backup this is you last line of defence. >Can your backups be accidentally destroyed? Well if someone "accidentally" breaks into my bank opens the right tresor and "accidentally" uses a flametrower..the you are right.
- jefftk 6y ago> since were able to restore all videos and all photos - but not all photos at the original resolution "Canon has said that there is no technical measure to restore lost video files, but that photo files can be restored – albeit not at their original resolution."
- virgilp 6y agoAh I somehow misread that, thought they can restore videos but for photos they don't have the original resolution.
- classified 6y agoThat. Who in their right mind would expect IT professionals to take any responsibility whatsoever for the empty promises they make to their users?
- labster 6y agoI’m sure plenty of engineers will be made redundant by this fiasco.
- jacquesm 6y agoAnd even then, if you store your data there the responsibility is ultimately still yours.
- StillBored 6y agoI suspect this is more a view from the trenches of "modern" SAS companies hosting their offerings on cloud providers charging a premium for storage+transit. Because I can assure you, from the perspective of someone who worked for >10 years in a backup related part of the storage industry, our product penetration covered a huge portion of public companies, and untold numbers of non public businesses. Given that we were one of a half dozen or so similar products and our competitors also had non-insignificant footprints in places we didn't most companies were taking backup of their core systems very seriously. And even then, today, there are dozens of online backup services. There are plans which can backup a TB or two of data a year off a NAS/etc and provide a read only historical snapshot for the price of a couple starbucks coffees a month (and a reasonable internet connection). And then there are the dozens of mom & pop places which just use two or three ~$100 USB disks on a rotation with windows backup/time machine/whatever. So, no, lack of a proper backup (and recovery) strategy isn't really a question of cost. Its more a question of the competencies of the individuals charged with configuring/maintaining the IT systems. And sure, if you have .5PB+ of data that needs backing up, a tape library, or replicating dedupe appliance, software license, etc is going to cost $$$$ but overwhelmingly when compared with the overall IT budget for machines+storage+administrators those devices rarely crack double digit percentages of the yearly IT budgets. Further, there are ways to lower those costs even more with a bit of hunting on the resale markets where picking up a three year old piece of equipment for a tiny fraction of its original cost isn't unusual.