5 ms·

In the meanwhile, you could grab an existing LTO-6 drive, and archive everything on 6 uncompressed tapes in about 30 hours (much less if your data is prone to c

by wazoox 13y ago

In the meanwhile, you could grab an existing LTO-6 drive, and archive everything on 6 uncompressed tapes in about 30 hours (much less if your data is prone to compression).

hga 13y ago
After building a very very fast staging disk system to feed it. That's one of many reasons I went with LTO-4 and a slower HP tape drive, I can feed it without shoe-shining from one 15K SAS disk that I have Bacula staging to ... and I don't care about how long it takes to do my backups or restores, which is generally not the case for businesses. Back when I made that choice, it was said no single magnetic disk could keep up with an LTO-5 tape drive.
beagle3 13y ago
Or 4x 4TB drives (at ~$180-$250 each from a quick newegg search), at ~15 hours if you can connect all of them at the same time, or ~60 hours if you need to do that serially (assuming 75MB/sec, which I often get - are you sure you're actually getting 150MB/sec on your LTO-6 tapes?) At about a quarter of the price of the LTO tape drive alone -- data can be recovered anywhere for the next 20 years or so (I have tapes from 2002 that I was unable to find a tape drive for in 2007, but I have a pair of 15GB IDE disks from 2000 that I checked last month and are still readable). At $60-$80/2.5TB, ($24/TB) LTO6 does have an advantage to disk ($40/TB) if you have hundreds of TBs to backup. However, because of the cost of the drive itself ($3000), you only break even after 200TB or so. And if your drive breaks (or you need to backup in one place and recover in another, as is often the case with DR plans), you break even only after 400TB or so. Since approximately 1998, hard drives for backup beat tapes on capacity, price, access speed (drives are random access!) accessibility (an external universal SATA/IDE can be bought for $30 or so, but you already have the connections on your motherboard so you don't even need it). I haven't been able to find anyone with a convincing argument for tapes - maybe you have one?
- hga 13y ago
  I'd be more interested if you made your case with LTO-5 tape; the latest and greatest generally comes at a price premium. I switched from USB 2 external hard drives to LTO-4 (see my other comment in this thread for one reason for that generation, which BTW per the LTO spec can be read by an LTO-6 drive) because I wanted multiple copies of my data in a less physically fragile form than hard drives (something I'm willing to routinely schlep to a safe deposit box). And I'm an old timer, bought my first DECtape in 1978 (sic), because it was neat, and then it became handy when I learned the -rf flags to the rm command ^_^. I'm very used to tapes, how they work and don't, what you can and can't do with them. And while I was flirting with hard disk backup almost had a catastrophic loss (http://www.ancell-ent.com/1715_Rex_Ave_127B_Joplin/images/ http://www.ancell-ent.com/1715_Rex_Ave_127B_Joplin/images/) only redeemed by luck and offsite storage to rsync.net in Denver. I didn't buy a high end, super fast tape drive, rather an internal HP 1760 EH919 drive, Newegg will sell you one at 2,130, but with some bargain hunting you can get for $1,400 easily (we managed $1,200), and Fujifilm and HP tapes (wouldn't trust any others, except maybe IBM but those are expensive) from them are going for $39 to $42 quantity one, you can of course get better pricing if you buy packs or 20 (or more) and/or go bargain hunting, e.g. 35.45 for the first Amazon Merchant I found with a rating >= 96% for a 20 pack of HP tapes. That's for 800GB uncompressed storage per tape. Per my other comment I also had to buy a 15K SAS drive, that was $250 back then (late summer of 2011). And I should note one of the biggest use cases for LTO drives is fast backups for businesses. With of course plenty of $$$ backing hardware to keep LTO-5 or LTO-6 drives happy.
  - beagle3 13y ago
    Case is not much different for LTO 5: drive is a little cheaper, as you note. Media is slower and almost same price per TB. You still need drive redundancy in most setups. And my experience with tapes has been that they are not significantly more robust than drives: they break and die when they fall on a hard floor, they get ruined if heated too much. And an issue that might be gone in the recent LTO era (has it?), but which I met several times: if you need to recover tapes made 5 years ago, finding a usable tape drive for use or sale might be a big problem. The last drive I had I can't connect to a modern motherboard was an MFM controller-RLL drive pair I had in 1987. IDE came shortly after.
- odonnellryan 13y ago
  Look here: http://www.clir.org/pubs/reports/pub54/estimation_of_LEs.html http://www.clir.org/pubs/reports/pub54/estimation_of_LEs.htm... Lifetime of tapes may not be what you think it is.
  - beagle3 13y ago
    10-20 years in my environment (60-95% humidity if not controlled) is what I expected. And that's what I've been getting from older left-in-storage drives since 2000 (all of them still ok).
  - hga 13y ago
    While I'd hope professional backup tape is a little better than high end early 1990s VHS tapes, the 10-20 years that beagle3 expects is also in line with what is generally expected out of modern backup tape. Granted, we are talking about archival storage, and it is still used for that, but after disks got big enough that tapes were seldom used for more than backups the pattern is to continually refresh them with multiple backups from disk.
    - odonnellryan 13y ago
      It may be better, or worse. But temp matters a lot - several data centers I've helped in are often left to get to almost 80 degrees - I can see that being a huge problem on their lifespan! I know from experience that people are very likely to think "Tape?! It'll last forever!...right?" which is dangerous.
- wazoox 13y ago
  Well I've recently restored 150 LTO-2 tapes from 2002/2003 using a perfectly current LTO-4 drive, and I'm pretty sure you'd have a heck of a hard time successfully reading all of a bunch of 150 hard drives of the same vintage without losing a single file. Hard drives are much less reliable than tapes; hard drives aren't made to be stored on shelves; and when a hard drive fails, you most of the time lose most or all of its data, while a tape failure generally affects only one file (if it's a reasonably large file and you aren't using compression). As soon as your storage and archival needs are more than a few disk drives, do yourself a service and switch either to RDX or to tape. Storing basic HDD on shelves is a recipe for data loss. I know it because we sell backup solutions to people doing it and losing data all the time :)
  - bluedino 13y ago
    >> Hard drives are much less reliable than tapes; hard drives aren't made to be stored on shelves; I can back this up. For some reason, someone decided to back things up to hard drives and even though they are stored in anti-static bags in a 600lb fire safe, when I remove the drives from 1 year ago to re-write with that months backup, I've had up to 1/2 the drives not spin back up. Ugh.
  - beagle3 13y ago
    Disclaimer: my experience with tapes is dated. I used IBM tapes with a mainframe (System 390, library robot and all) until 1995, and then only used consumer level myself, but had witnessed the occasional horror story at a client's. > Well I've recently restored 150 LTO-2 tapes from 2002/2003 using a perfectly current LTO-4 drive, and I'm pretty sure you'd have a heck of a hard time successfully reading all of a bunch of 150 hard drives of the same vintage without losing a single file. That might be true. However, I have so far had perfect success with about 50 or so drives that I've restored from (across 10 years or so), and abysmal success with tapes (about 1/10 in the mainframe days would not restore). > Hard drives are much less reliable than tapes; hard drives aren't made to be stored on shelves; That's true, but neither are tapes. Hard drives are less tolerant to environmental conditions than tapes, they survive better when exposed to heat, and get damaged more quickly when exposed to cold (oils used to keep it running smooth tend to congeal irreversibly when exposed to cold for long). Also, at least in the past, even weak magnetic fields wreaked havoc on tapes, and even strong ones spared drives. > As soon as your storage and archival needs are more than a few disk drives, do yourself a service and switch either to RDX or to tape. Storing basic HDD on shelves is a recipe for data loss. I know it because we sell backup solutions to people doing it and losing data all the time :) In my current top-secret venture, I need to keep everything (~20TB/customer) randomly accessible for 3 years, and cold-store accessible or 7. So far, we just keep multiple copies (some online and some offline), and keep refershing to newer larger drives every couple of years. When I specced it 3 years ago, tapes were twice as expensive (I need a tape drive at every customer location + at least two at my office) and didn't properly address the random access aspect. But I keep re-evaluating. So far, the next step seems to be going to a BackBlaze style pod - which would also address the valid drive-fails-completely concern you raised. p.s., I find the "compressed" statistics really misleading. Every kind of backup software can compress - if I saw a hard drive manufacturer sell a ("2TB drive (1TB uncompressed)" I would consider it fraudulent, but that's the standard for tapes. For the record, my data is (as far as tape drives are concerned) not compressible at all -- think audio, video, jpegs, and other analog recordings. EDIT: p.p.s: Of course, don't keep hard drives on shelves. Nor tapes. Humidity and temperature need to be controlled, and the environment needs to be anti-static. For archival purposes, the 5400-RPM usb self-powered disks are a little slow, but keep very well, in my experience, with reasonable temp/humidity. EDIT2: (bad) memories are coming back. The biggest problem I recall with tapes in the mainframe days, were that some tapes (the daily and weekly backup sets) were continuously reused, to the point that mechanical wear and tear was a bigger problem than magnetic wear. And the even bigger problem was that you only found that out when trying to restore. One advantage of disks is that you keep reading them while writing them - wear is apparent much more quickly. If you're only doing archives and never rewrite tapes, that doesn't matter. If you keep reusing the same tapes/disks, it makes a huge difference in reliability.

by wazoox 13y ago

In the meanwhile, you could grab an existing LTO-6 drive, and archive everything on 6 uncompressed tapes in about 30 hours (much less if your data is prone to compression).

hga 13y ago

After building a very very fast staging disk system to feed it. That's one of many reasons I went with LTO-4 and a slower HP tape drive, I can feed it without shoe-shining from one 15K SAS disk that I have Bacula staging to ... and I don't care about how long it takes to do my backups or restores, which is generally not the case for businesses. Back when I made that choice, it was said no single magnetic disk could keep up with an LTO-5 tape drive.

beagle3 13y ago

Or 4x 4TB drives (at ~$180-$250 each from a quick newegg search), at ~15 hours if you can connect all of them at the same time, or ~60 hours if you need to do that serially (assuming 75MB/sec, which I often get - are you sure you're actually getting 150MB/sec on your LTO-6 tapes?) At about a quarter of the price of the LTO tape drive alone -- data can be recovered anywhere for the next 20 years or so (I have tapes from 2002 that I was unable to find a tape drive for in 2007, but I have a pair of 15GB IDE disks from 2000 that I checked last month and are still readable). At $60-$80/2.5TB, ($24/TB) LTO6 does have an advantage to disk ($40/TB) if you have hundreds of TBs to backup. However, because of the cost of the drive itself ($3000), you only break even after 200TB or so. And if your drive breaks (or you need to backup in one place and recover in another, as is often the case with DR plans), you break even only after 400TB or so. Since approximately 1998, hard drives for backup beat tapes on capacity, price, access speed (drives are random access!) accessibility (an external universal SATA/IDE can be bought for $30 or so, but you already have the connections on your motherboard so you don't even need it). I haven't been able to find anyone with a convincing argument for tapes - maybe you have one?

hga 13y ago
I'd be more interested if you made your case with LTO-5 tape; the latest and greatest generally comes at a price premium. I switched from USB 2 external hard drives to LTO-4 (see my other comment in this thread for one reason for that generation, which BTW per the LTO spec can be read by an LTO-6 drive) because I wanted multiple copies of my data in a less physically fragile form than hard drives (something I'm willing to routinely schlep to a safe deposit box). And I'm an old timer, bought my first DECtape in 1978 (sic), because it was neat, and then it became handy when I learned the -rf flags to the rm command ^_^. I'm very used to tapes, how they work and don't, what you can and can't do with them. And while I was flirting with hard disk backup almost had a catastrophic loss (http://www.ancell-ent.com/1715_Rex_Ave_127B_Joplin/images/ http://www.ancell-ent.com/1715_Rex_Ave_127B_Joplin/images/) only redeemed by luck and offsite storage to rsync.net in Denver. I didn't buy a high end, super fast tape drive, rather an internal HP 1760 EH919 drive, Newegg will sell you one at 2,130, but with some bargain hunting you can get for $1,400 easily (we managed $1,200), and Fujifilm and HP tapes (wouldn't trust any others, except maybe IBM but those are expensive) from them are going for $39 to $42 quantity one, you can of course get better pricing if you buy packs or 20 (or more) and/or go bargain hunting, e.g. 35.45 for the first Amazon Merchant I found with a rating >= 96% for a 20 pack of HP tapes. That's for 800GB uncompressed storage per tape. Per my other comment I also had to buy a 15K SAS drive, that was $250 back then (late summer of 2011). And I should note one of the biggest use cases for LTO drives is fast backups for businesses. With of course plenty of $$$ backing hardware to keep LTO-5 or LTO-6 drives happy.
- beagle3 13y ago
  Case is not much different for LTO 5: drive is a little cheaper, as you note. Media is slower and almost same price per TB. You still need drive redundancy in most setups. And my experience with tapes has been that they are not significantly more robust than drives: they break and die when they fall on a hard floor, they get ruined if heated too much. And an issue that might be gone in the recent LTO era (has it?), but which I met several times: if you need to recover tapes made 5 years ago, finding a usable tape drive for use or sale might be a big problem. The last drive I had I can't connect to a modern motherboard was an MFM controller-RLL drive pair I had in 1987. IDE came shortly after.
odonnellryan 13y ago
Look here: http://www.clir.org/pubs/reports/pub54/estimation_of_LEs.html http://www.clir.org/pubs/reports/pub54/estimation_of_LEs.htm... Lifetime of tapes may not be what you think it is.
- beagle3 13y ago
  10-20 years in my environment (60-95% humidity if not controlled) is what I expected. And that's what I've been getting from older left-in-storage drives since 2000 (all of them still ok).
- hga 13y ago
  While I'd hope professional backup tape is a little better than high end early 1990s VHS tapes, the 10-20 years that beagle3 expects is also in line with what is generally expected out of modern backup tape. Granted, we are talking about archival storage, and it is still used for that, but after disks got big enough that tapes were seldom used for more than backups the pattern is to continually refresh them with multiple backups from disk.
  - odonnellryan 13y ago
    It may be better, or worse. But temp matters a lot - several data centers I've helped in are often left to get to almost 80 degrees - I can see that being a huge problem on their lifespan! I know from experience that people are very likely to think "Tape?! It'll last forever!...right?" which is dangerous.
wazoox 13y ago
Well I've recently restored 150 LTO-2 tapes from 2002/2003 using a perfectly current LTO-4 drive, and I'm pretty sure you'd have a heck of a hard time successfully reading all of a bunch of 150 hard drives of the same vintage without losing a single file. Hard drives are much less reliable than tapes; hard drives aren't made to be stored on shelves; and when a hard drive fails, you most of the time lose most or all of its data, while a tape failure generally affects only one file (if it's a reasonably large file and you aren't using compression). As soon as your storage and archival needs are more than a few disk drives, do yourself a service and switch either to RDX or to tape. Storing basic HDD on shelves is a recipe for data loss. I know it because we sell backup solutions to people doing it and losing data all the time :)
- bluedino 13y ago
  >> Hard drives are much less reliable than tapes; hard drives aren't made to be stored on shelves; I can back this up. For some reason, someone decided to back things up to hard drives and even though they are stored in anti-static bags in a 600lb fire safe, when I remove the drives from 1 year ago to re-write with that months backup, I've had up to 1/2 the drives not spin back up. Ugh.
- beagle3 13y ago
  Disclaimer: my experience with tapes is dated. I used IBM tapes with a mainframe (System 390, library robot and all) until 1995, and then only used consumer level myself, but had witnessed the occasional horror story at a client's. > Well I've recently restored 150 LTO-2 tapes from 2002/2003 using a perfectly current LTO-4 drive, and I'm pretty sure you'd have a heck of a hard time successfully reading all of a bunch of 150 hard drives of the same vintage without losing a single file. That might be true. However, I have so far had perfect success with about 50 or so drives that I've restored from (across 10 years or so), and abysmal success with tapes (about 1/10 in the mainframe days would not restore). > Hard drives are much less reliable than tapes; hard drives aren't made to be stored on shelves; That's true, but neither are tapes. Hard drives are less tolerant to environmental conditions than tapes, they survive better when exposed to heat, and get damaged more quickly when exposed to cold (oils used to keep it running smooth tend to congeal irreversibly when exposed to cold for long). Also, at least in the past, even weak magnetic fields wreaked havoc on tapes, and even strong ones spared drives. > As soon as your storage and archival needs are more than a few disk drives, do yourself a service and switch either to RDX or to tape. Storing basic HDD on shelves is a recipe for data loss. I know it because we sell backup solutions to people doing it and losing data all the time :) In my current top-secret venture, I need to keep everything (~20TB/customer) randomly accessible for 3 years, and cold-store accessible or 7. So far, we just keep multiple copies (some online and some offline), and keep refershing to newer larger drives every couple of years. When I specced it 3 years ago, tapes were twice as expensive (I need a tape drive at every customer location + at least two at my office) and didn't properly address the random access aspect. But I keep re-evaluating. So far, the next step seems to be going to a BackBlaze style pod - which would also address the valid drive-fails-completely concern you raised. p.s., I find the "compressed" statistics really misleading. Every kind of backup software can compress - if I saw a hard drive manufacturer sell a ("2TB drive (1TB uncompressed)" I would consider it fraudulent, but that's the standard for tapes. For the record, my data is (as far as tape drives are concerned) not compressible at all -- think audio, video, jpegs, and other analog recordings. EDIT: p.p.s: Of course, don't keep hard drives on shelves. Nor tapes. Humidity and temperature need to be controlled, and the environment needs to be anti-static. For archival purposes, the 5400-RPM usb self-powered disks are a little slow, but keep very well, in my experience, with reasonable temp/humidity. EDIT2: (bad) memories are coming back. The biggest problem I recall with tapes in the mainframe days, were that some tapes (the daily and weekly backup sets) were continuously reused, to the point that mechanical wear and tear was a bigger problem than magnetic wear. And the even bigger problem was that you only found that out when trying to restore. One advantage of disks is that you keep reading them while writing them - wear is apparent much more quickly. If you're only doing archives and never rewrite tapes, that doesn't matter. If you keep reusing the same tapes/disks, it makes a huge difference in reliability.