7 ms·
I wish tape drives weren't so expensive. I'd love to have on each personal computer a backup system that just mirrors every disk write to tape. According to th
by tzs 5y ago
I wish tape drives weren't so expensive. I'd love to have on each personal computer a backup system that just mirrors every disk write to tape.
According to the SMART data for my current system I write about 9 TB per year. That would fit on one LTO 8 cartridge.
- klodolph 5y agoWhen I last calculated the break-even point for a thrifty home user, it was around 50TB of data. Above 50TB and there was a decent chance that tape is cheaper than disk, below 50TB it's unlikely. I'll just say that the exact tradeoff will depend on a lot of factors, it's just as a rule of thumb, you can forget about tape below about 50TB. The media itself is cheap and durable, but the drives are $1,000 or more and sometimes break--in other words, if you really must have your data, you want two drives.
- userbinator 5y agoin other words, if you really must have your data, you want two drives. On the bright side, since it's removable media, anyone with a working drive can read your tapes for you.
- ineedasername 5y agoDepends on the needs. Tape will keep longer unpowered. A traditional HDD is a bit riskier if left offline and unpowered for a while. Even if they're all powered in a tower, I'd still recommend something like Backblaze for redundancy.
- klodolph 5y agoIf you're storing 50TB of data in B2, that's $3k/year. For that price, you can make 2x copies of your data on tape, and then buy two tape drives to read them.
- ineedasername 5y agoB2 is an upgrade option. You can still store an unlimited amount on their standard plan, or upgrade for a little bit more for access to a year's worth of changes. It would be worth it to keep a separate desktop with a dozen drives connected and pay for a second annual subscription if you have that much data. The one-year retention plan adds $24/year. Even the "forever" retention plan for 50TB would only add $250/year. If we're talking about home use, B2 really isn't necessary. And if you want a daily rotation every day for a year the tape costs can add up, depending on how much data Cha he's day to day. For home use, an HDD backup combined with something like Backblaze seems like a pretty reasonable cost for two layers of redundancy and a year's worth of file versions changes.
- klodolph 5y agoKind of a hypothetical scenario for me—backing up 50TB to cloud would saturate my uplink for a full year and more. By comparison, it’s about four days of drive time on LTO-5. So while LTO-5 is a plausible way for me to store 50TB of data, cloud storage is decidedly a fantasy. Personally—gonna say I don’t trust arrangements where I know the other side is losing money on the deal. This includes “unlimited” storage. I put “unlimited” in quotes because all of these other services with unlimited storage have, at some point, gotten shut down their unlimited storage plans. That’s why I compare to B2… not really interested in gambling on those things. So I would take B2 as the baseline for comparison when you're talking about this much data. Amazon once had an unlimited storage option, if you recall. One data hoarder was recording cam girl streams and had just north of 1 PB in his Amazon Drive folder. Whole thing got shut down shortly after.
- ineedasername 5y agoBandwidth limitation is a great example of how specific circumstances should dictate backup protocol. Tape absolutely makes sense there even in a home environment. In your case, if it was really critical, I'd probably want to get two tapes at certain intervals so I could ship a copy off site and have the off-site copy online in a dedicated server at a data center so I could retrieve critical files if needed. Probably too extreme for a home setup unless it's a home business, but I've also had to restore from an overnight tape when a database went corrupt (multiple times) so if I was still involved in that sort of sys admin work I'd be a bit paranoid. As for financial issues with "unlimited", Backblaze has a roughly 15 year track record and has been profitable for almost as long. They're not a typical startup looking to pump up a customer base by selling below costs and then dump to an exit acquisition or IPO. Storage is cheap, and they have managed to build a business that doesn't screw up the economics of low marginal costs by becoming overly bloated. That said, they're still one of of two layers of redundancy I use for my home backups. I've seen cloud services lose files, so I'm banking on the fact that my own storage won't go down at the same time as two cloud services.
- zozbot234 5y ago> The media itself is cheap and durable, but the drives are $1,000 or more and sometimes break--in other words, if you really must have your data, you want two drives. Newer drives will also drop support for older LTO standards, so you end up having to rely on flimsy old hardware to recover your older tapes. For most home users, it just doesn't seem worth the hassle. Maybe it starts making sense for casual/non-professional users in the 100TB+ range, but even the most committed data hoarders are nowhere near that.
- ClumsyPilot 5y ago"even the most committed data hoarders are nowhere near that." Eh, i do meet people with ~50TB archives
- jacquesm 5y ago> but even the most committed data hoarders are nowhere near that Not all that committed, well over that.
- justahuman74 5y agoWhat's the ball park cost on a drive+tape (say for a desktop)?
- klodolph 5y agoKind of like asking the cost of a car + groceries. You can pick up an old tape drive, like an LTO5, for about $1,000. You can LTO-5 tapes for about $15 each, with a capacity of 1.5 TB. This setup puts the cost at around $10/TB, compared to somewhere around $30/TB for disk. Since you save $20/TB by using tape, it takes you 50TB before the total setup is less expensive than disk. Plug in your own numbers if you like. You'll want to get a fiber channel card too, but that's easy enough. Newer generations of LTO have marginally cheaper media (per TB) when you're buying in small quantities, but the drives are more expensive, and it's not uncommon to see poor capacity utilization in tape systems. The big players with lots of data will get newer LTO generations because they can save on library space, vault space, and drive time.
- bombcar 5y agoIf you drop back to used LTO4 you can get an entire setup for less than $500.
- klodolph 5y agoWhich is great, until the heads wear out. I would hesitate to buy a used drive unless the heads have been replaced, especially since LTO-4 is about 14 years old at this point.
- bombcar 5y agoGood point - and to be really secure you should have two drives - write on one and read back and verify on the other.
- cameldrv 5y agoHowever, if you buy external USB hard drives, it's only about $20/TB, you have full random access, you don't need a fiber channel card, and if you buy a bunch of USB hubs, you don't have to change tapes every 1.5 TB. IMO the niche where tape still has an advantage is pretty small these days.
- ineedasername 5y agoThey used to be a lot cheaper. When I did admin work for a small company ~2004 they weren't as cheap as a DVD R/W, but they were still pretty inexpensive, and I kept rotating 7-day backups on them along with a monthly rotation, and IIRC the system itself was RAID 3. When I upgraded the server I think the DAT tape drive, 4GB per tape, only added a a few hundred dollars. These days a new tape drive goes into the thousands. It's a shame because it's still very difficult to beat tape for high capacity unpowered offline storage. Some optical formats have longer shelf lives but lower capacities, and also aren't cheap. I'm glad I don't have to worry about backup/restore in my job anymore. I had to make use of those daily tapes more than once and the stress of that was not fun.
- srtjstjsj 5y agoHow would you make use of data that was "every write", which could just be blocks of subfiles?
- tzs 5y agoYou can restore the state of the drive to an arbitrary time in the past by replaying past writes. For example, suppose something bad happens (a bug deletes important files, someone installs ransomware, etc) and you want to roll back the system three days. Restore the last full backup you have that is older than the desired rollback time. Then replay from the write log every write that occurred between the time of that backup and the time you are trying to rollback to. Another use for it is restoring or reverting files that were originally created more recently than your last regular (full or incremental) backup. Example: I do incremental cloud backups once a week, on Saturday. Suppose I create a file on Monday, make various changes to it on Tuesday and Wednesday, accidentally delete it or corrupt it on Thursday, and realize this on Friday and want to recover the file. With the "every write" log, I could find the file in the write log and recover it. You might at first think that I would not be able to find it because unlike most backup programs the write log does not explicitly record file information. But it doesn't have to explicitly record file information because when you create a file or modify it the OS stores that information by writing to the disk, and those writes end up in your tape write log. If I'm trying to restore /home/tzs/important_file to the state it was two days ago, the restore process would scan the write log starting far enough back to find the latest earlier write of the root inode (let's assume it was a classic Unix filesystem). Once it has that it can find /home's inode, then can find the correct contents of that on the target date, and so on until it has the inode of important_file. Then it can find what blocks comprised the file, and pull the right versions of those blocks from the write log. Note that for this to work, the restore program must intimately understand the filesystem whose write log it is working with. This is one of the reasons you want to still do a more normal full/incremental period backup system with a backup program that explicitly stores files. The write log is best used to supplement the traditional approach. PS: there was a backup program in whatever AT&T Unix was current in the early '80s (System III) that used the write log approach, although it was not continuous and it wrote redundant data. It was used for incremental backups. You started with a full block level backup. Then when you wanted to do an incremental backup it did this: 1. Make an empty list of blocks. 2. Add the block number of the superblock to the list. 3. Scan all the inodes looking for any inode that has a create or modify time later than the previous backup. For any such inode: 3a. Add the number of the block that contains that inode to the list. 3b. Add the block numbers of the file that the inode refers to to the list, including any indirect blocks. 4. Sort the block number list. 5. Copy all the blocks that are referenced in the list to the tape. (There was probably a "write some metadata about the backup to the tape" step in there somewhere, too). Note that this approach was redundant, in that if you changed part of a file the whole file would be written to the incremental backup. A pure write log would only write the changed blocks, but most systems don't record sufficiently granular modification information to allow that. On the plus side, it meant that restoring a file from an incremental backup was easy. The file was either in that incremental entirely or not at all.
- matheusmoreira 5y agoIt's not just the price either. LTO has backwards compatibility for the previous two generations only. There's a risk data stored on LTO cartridges will not be readable by future tape drives.
- johnklos 5y agoUsed tape drives with not many hours on them are easy to find on eBay and other places for a fraction of the cost of new drives, and LTO-5 and 6 tapes, for instance, are pretty inexpensive.