5 ms·

If you need the following: - Storing petabytes of data without requiring any power. Hard-drives are cheap but each one will require quite a bit of power - Hav

by perbu 3y ago

If you need the following: - Storing petabytes of data without requiring any power. Hard-drives are cheap but each one will require quite a bit of power - Having the data be offline, ie, not available to an attacker which somehow managed to take control over your network - Having the data be in a physical format which can be easily put in a safe or in the back of a car when there is a risk of flood or fire - Having data on a media that won't regrade over a period of say, 10-30 years. Hard-drives tend to fail

izzydata 3y ago
What stops you from writing data to a spinning disk HDD and then unplugging it and putting it in a safe?
- comboy 3y ago
  Typical reliable offline HDD storage is 2-3 years. For tape it's 20-30 years.
- jdhawk 3y ago
  Bitrot.
- perbu 3y ago
  I've got a bunch of video on a harddrive that I put in storage 20 years ago. Tried to spin it up again and it just makes funny noises. So, yeah, spinning rust might rust.
  - seniorivn 3y ago
    its possible to restore it in a data restoration shop, if you are interested
- fbnlsr 3y ago
  This is what I personally do. I have small needs when it comes to backups (< 1TB), so I have one backup that's just a regular HDD stored in a drawer, and another one that's on a Scaleway cold storage container.
- throw0101a 3y ago
  > What stops you from writing data to a spinning disk HDD and then unplugging it and putting it in a safe? Manual labour. There are tape libraries with robots that can take a tape out of a slot, put it in a drive, and then put it back automatically without humans needing to do anything. If you want to move the tapes offsite, there is some manual work to move the tapes from the import/export slot to a container. A large library like the StorageTek SL8500 can have up to 640 drives, 100'000 tape storage slots, and 2'880 import/export slots.
  - sofixa 3y ago
    > Manual labour. Big cloud providers have disk replacement robots, so it's not unfeasible for robots to perform the "unplug and store the HDD somewhere". But another advantage tape has is durability.
    - jeffbee 3y ago
      Who actually has those?
      - jandrese 3y ago
        Last time I saw one of those was back when AOL was a big shot ISP. They had to buy high power SGI systems to feed the things because nothing else had the bus bandwidth to support one. I was told that the full rack sized SGI system (Origin 2000) was the cheapest part of the entire rig.
      - deleted 3y ago
        [deleted]
    - throw0101a 3y ago
      > Big cloud providers have disk replacement robots […] [citation needed]
      - voxadam 3y ago
        https://twitter.com/alibaba_cloud/status/1326078390105825281 https://twitter.com/alibaba_cloud/status/1326078390105825281 I have no idea if anything like this is actually in use, I have my doubts. Also, a system like this probably isn't terribly useful, a Cartesian gantry robot would be much faster and way more robust than an articulated arm strapped to a mobile robot platform.
- saltminer 3y ago
  Mechanical failure [0]. As perbu found out, when moving parts are left to sit for years, things can fail. Lube can dry up, causing bearings to seize up, actuators to lose precision or get completely stuck [1], and the magnetic field degrades by 1% per year [2]. Of course, you could get lucky w/r/t mechanical issues, as many retro computer enthusiasts can attest, but it seems to involve a lot of luck, as it only takes tiny defects to cause problems over a long enough time-frame that leave half your drives as paperweights while the others keep on chugging. Both of these issues can be largely mitigated by simply plugging the HDDs in every year or so and re-writing the data (maybe run badblocks while you're at it to ensure the drive is still in good health), but, as throw0101a mentioned, this takes time and effort. [0]: https://serverfault.com/a/51893 https://serverfault.com/a/51893 [1]: https://www.partitionwizard.com/clone-disk/do-hard-drives-go-bad-if-not-used.html https://www.partitionwizard.com/clone-disk/do-hard-drives-go... [2]: https://superuser.com/a/312764 https://superuser.com/a/312764
  - 3-cheese-sundae 3y ago
    Is there a way to force a file system to recommit its data?
    - jandrese 3y ago
      Seems like you could do it by not even mounting the filesystem, just read out raw sectors and write them back. Like a 3 line program. Maybe if you want to be fancy you compare the sectors to some checksums to make sure you're not reading corrupt data.
  - e12e 3y ago
    Any word on SSDs?
    - saltminer 3y ago
      SSDs may actually worse than HDDs, depending on what NAND chips are in use. Although there are no moving parts to seize up, they do require an occasional power up because they represent data via voltage levels and we have not yet found a perfect insulator (meaning they will inevitably leak charge), and with QLC being all the rage these days (4 bits represented in each cell using one of 16 discrete voltage levels), when the voltage levels start to degrade, it could easily cause catastrophic bit rot. How long a given SSD will last before losing data depends on the chips it is built with. I remember reading that Sandisk rates their flash storage as being able to survive a year without power (though I read that years ago and cannot find a source for it now), though [0] says 2-5 years, so technology has likely advanced since I last read about this. Either way, you should probably try and power up your SSDs at least once a year to keep the cells charged. This is less work than to keep an HDD healthy for cold storage, but it certainly isn't "set and forget" like tapes or optical disks. Edit: I remember pulling my first SSD, a 60 GB Intel 330, from an old laptop and, despite having not been powered on for 3 years, it still booted. But that was MLC/DLC [1], which only has 4 voltage states, and I only saw that it booted, not that all my old data was intact. [0]: https://www.easeus.com/resource/does-ssd-need-power.html https://www.easeus.com/resource/does-ssd-need-power.html [1]: https://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/ssd-330-specification.pdf https://www.intel.com/content/dam/www/public/us/en/documents...
- hulitu 3y ago
  The spinning motor might not start after some years of inactivity.
Dylan16807 3y ago
I agree with most of that, but taking a server offline isn't really harder than pulling tapes out of a library. And if you replace dead disks every month or two, a non-networked server can probably manage more reliability over 10-30 years.

by perbu 3y ago

izzydata 3y ago

What stops you from writing data to a spinning disk HDD and then unplugging it and putting it in a safe?

comboy 3y ago
Typical reliable offline HDD storage is 2-3 years. For tape it's 20-30 years.
jdhawk 3y ago
Bitrot.
perbu 3y ago
I've got a bunch of video on a harddrive that I put in storage 20 years ago. Tried to spin it up again and it just makes funny noises. So, yeah, spinning rust might rust.
- seniorivn 3y ago
  its possible to restore it in a data restoration shop, if you are interested
fbnlsr 3y ago
This is what I personally do. I have small needs when it comes to backups (< 1TB), so I have one backup that's just a regular HDD stored in a drawer, and another one that's on a Scaleway cold storage container.
throw0101a 3y ago
> What stops you from writing data to a spinning disk HDD and then unplugging it and putting it in a safe? Manual labour. There are tape libraries with robots that can take a tape out of a slot, put it in a drive, and then put it back automatically without humans needing to do anything. If you want to move the tapes offsite, there is some manual work to move the tapes from the import/export slot to a container. A large library like the StorageTek SL8500 can have up to 640 drives, 100'000 tape storage slots, and 2'880 import/export slots.
- sofixa 3y ago
  > Manual labour. Big cloud providers have disk replacement robots, so it's not unfeasible for robots to perform the "unplug and store the HDD somewhere". But another advantage tape has is durability.
  - jeffbee 3y ago
    Who actually has those?
    - jandrese 3y ago
      Last time I saw one of those was back when AOL was a big shot ISP. They had to buy high power SGI systems to feed the things because nothing else had the bus bandwidth to support one. I was told that the full rack sized SGI system (Origin 2000) was the cheapest part of the entire rig.
    - deleted 3y ago
      [deleted]
  - throw0101a 3y ago
    > Big cloud providers have disk replacement robots […] [citation needed]
    - voxadam 3y ago
      https://twitter.com/alibaba_cloud/status/1326078390105825281 https://twitter.com/alibaba_cloud/status/1326078390105825281 I have no idea if anything like this is actually in use, I have my doubts. Also, a system like this probably isn't terribly useful, a Cartesian gantry robot would be much faster and way more robust than an articulated arm strapped to a mobile robot platform.
saltminer 3y ago
Mechanical failure [0]. As perbu found out, when moving parts are left to sit for years, things can fail. Lube can dry up, causing bearings to seize up, actuators to lose precision or get completely stuck [1], and the magnetic field degrades by 1% per year [2]. Of course, you could get lucky w/r/t mechanical issues, as many retro computer enthusiasts can attest, but it seems to involve a lot of luck, as it only takes tiny defects to cause problems over a long enough time-frame that leave half your drives as paperweights while the others keep on chugging. Both of these issues can be largely mitigated by simply plugging the HDDs in every year or so and re-writing the data (maybe run badblocks while you're at it to ensure the drive is still in good health), but, as throw0101a mentioned, this takes time and effort. [0]: https://serverfault.com/a/51893 https://serverfault.com/a/51893 [1]: https://www.partitionwizard.com/clone-disk/do-hard-drives-go-bad-if-not-used.html https://www.partitionwizard.com/clone-disk/do-hard-drives-go... [2]: https://superuser.com/a/312764 https://superuser.com/a/312764
- 3-cheese-sundae 3y ago
  Is there a way to force a file system to recommit its data?
  - jandrese 3y ago
    Seems like you could do it by not even mounting the filesystem, just read out raw sectors and write them back. Like a 3 line program. Maybe if you want to be fancy you compare the sectors to some checksums to make sure you're not reading corrupt data.
- e12e 3y ago
  Any word on SSDs?
  - saltminer 3y ago
    SSDs may actually worse than HDDs, depending on what NAND chips are in use. Although there are no moving parts to seize up, they do require an occasional power up because they represent data via voltage levels and we have not yet found a perfect insulator (meaning they will inevitably leak charge), and with QLC being all the rage these days (4 bits represented in each cell using one of 16 discrete voltage levels), when the voltage levels start to degrade, it could easily cause catastrophic bit rot. How long a given SSD will last before losing data depends on the chips it is built with. I remember reading that Sandisk rates their flash storage as being able to survive a year without power (though I read that years ago and cannot find a source for it now), though [0] says 2-5 years, so technology has likely advanced since I last read about this. Either way, you should probably try and power up your SSDs at least once a year to keep the cells charged. This is less work than to keep an HDD healthy for cold storage, but it certainly isn't "set and forget" like tapes or optical disks. Edit: I remember pulling my first SSD, a 60 GB Intel 330, from an old laptop and, despite having not been powered on for 3 years, it still booted. But that was MLC/DLC [1], which only has 4 voltage states, and I only saw that it booted, not that all my old data was intact. [0]: https://www.easeus.com/resource/does-ssd-need-power.html https://www.easeus.com/resource/does-ssd-need-power.html [1]: https://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/ssd-330-specification.pdf https://www.intel.com/content/dam/www/public/us/en/documents...
hulitu 3y ago
The spinning motor might not start after some years of inactivity.

Dylan16807 3y ago

I agree with most of that, but taking a server offline isn't really harder than pulling tapes out of a library. And if you replace dead disks every month or two, a non-networked server can probably manage more reliability over 10-30 years.