7 ms·

> me wonder about their redundancy and recovery architecture You can pretty much guarantee it's untested and unlikely to work, when someone doesn't understand

• dikei 3y ago
  Yeah, I wouldn't trust their process at all, if they'd rather have the server down for 1 week instead of aborting and performing a rollback to the last known good state.
  • x0x0 3y ago
    I suspect they can't. Abort and rollback, that is. Evidence: it's not rational to eat a week of downtime if you have an alternative. edit: Which tells you something about their backups, and it ain't great. Or they'd restore onto another box.
• TylerE 3y ago
  This is why I ultimately trust my personal data to Apple (and to a lesser and smaller extent, Google). I don't expect either to exactly have my best interest at heart - but I do expect both of them to have basic ops competency.
  • istjohn 3y ago
    If you have multiple backups, you don't have to trust anyone.
    • greenie_beans 3y ago
      except yourself !
  • crossroadsguy 3y ago
    Actually if I must entrust my data to companies like these where reliability and competence is of utmost importance - I’d rather trust Google and Facebook (much much before Apple). If we are talking about privacy (or rather privacy theatre) then yeah Apple, why not.
    • chillfox 3y ago
      I pulled down my Google Drive data some time ago with rclone and a bunch of the older files were corrupted.
  • ksec 3y ago
    I wish Apple made a Time Capsule that requires iCloud 2TB Subscription to work. They could have priced it at $299 to include the initial 12 months iCloud subscription.
    • Wowfunhappy 3y ago
      Sorry, can you expand on that? Why require the sub?
      • ksec 3y ago
        Old Time Capsule isn't really a backup. You either need two HDD with BTRFS or ZFS set up as minimum. And then You will need an offsite backup option. iCloud , Google storage etc. If you only go via the iCloud route, you have zero local copy of your data. If you only only go via local, it isn't safe enough option. So ending up with a combination of the two. And Apple really wanted services revenue, if it was only just a Time Capsule I doubt Apple would ever make it again.
  • adobrawy 3y ago
    Large corporations have the technical skills in place, but organizationally they can lock your account away leaving you with no access and no real avenue to evict your data. There are many posts on HN and on the Internet where corporate support had no procedure to give back access to data when they blocked the account. Account bans are sometimes for unclear or out of control reasons (e.g. someone stole your credit card and then used that card with a another account, so the corporate banned all accounts indefinitely). In the case of a smaller organization, you have a better chance that if they have to say goodbye to you, they will do it without unnecessary harm to you.
    • aborsy 3y ago
      That would also be a major security concern: people gaining access to accounts by social engineering .
      • tjoff 3y ago
        Well, you do at least encrypt your backups yourself, right?
        • aborsy 3y ago
          That doesn’t prevent the deletion of backups and snapshots, encrypting the encrypted backups for a ransom, and using your account for distribution of the illegal content or malware.
          • tjoff 3y ago
            Sure, but lets try and be a little more realistic and weigh the scenarios accordingly.
• jm4 3y ago
  They messed up and either ran out of disk space or had an array fail. Everything in that statement reads like they are trying to frame a critical production failure as a glitch that occurred during maintenance. This wasn’t about a routine storage expansion. It was an emergency caused by some operational screwup and the only way to fix it is to rebuild the array with new disks.
  • jskrablin 3y ago
    I had a raid6 array going almost bad when 2 HDDs called it a day at the same time years ago. Had to send the entire office of 30 people to unscheduled holiday for two days before the array was rebuilt with new drives. Third drive failing during the rebuild would likely result in a week off as the storage would need a full rebuild. I can imagine something similar happening here, however there should be a fail over storage setup in case they're selling a service.
• viraptor 3y ago
  They messed up here in some way, but assuming you know what events and complaining about their skills is just making stuff up at this point. Unless you can tell us exactly what happened, maybe hold off with that kind of criticism.
  • colechristensen 3y ago
    > However, we did not anticipate that the expansion process would require the server storage to be temporarily offline. One, it is obviously some flavor of raid rebalancing, there’s not a different kind of thing it could be. Two, they’re admitting they didn’t expect this, that’s not possible if it’s a tested procedure.
    • x0x0 3y ago
      And three, they're admitting they don't back anything up (backup defined as not just slinging bytes somewhere but the availability to restore those bytes and bring a service back up on a timeline that is certainly well lower than 3 days. With regular testing to understand the timeline and ensure everything actually works.) ps -- it's not about being bashing these folks. But these type of multiple operational incompetencies strongly call into question their ability to safeguard your bytes and produce them when you need them. It's kinda the whole point of the thing.
      • wmf 3y ago
        It's a backup company. I can understand that they don't back up the backups.
        • colechristensen 3y ago
          It’s a backup company so I would expect the store 1 copy plus the minimum redundancy to make drive failures a predictable amount of unlikely to cause data loss. More than that would be additional features.
        • justinclift 3y ago
• boring_twenties 3y ago
  In a storage business no less. They literally had one job: to know how to resize arrays, so the rest of us don't have to.