11 ms·
50TB IBM tape drive more than doubles LTO-9 capacity
- vlovich123 3y agoWhat’s the point of reporting compressed capacity? Isn’t data typically compressed at rest, negating any value of the disk doing it?
- nikanj 3y agoBigger number for marketing
- vardump 3y agoPurely legacy, that's how it's always been reported.
- csydas 3y agoAfter having helped on a lot of tape storage implementations, I get why it's useful for quick estimates on backup solutions. Compression really is a game changer when you're over a certain amount of data you need to protect, and backup technology does some really clever space savings at basically all stages, making it hard to find a good and easy metric to help sell to management. It's a lot easier to tell management: "These tapes hold 50 TB of production data, and we have about 45 TB in production. Backups and restores will be faster, we'll have fewer tapes to manage for backups" than it is to explain the different types of data you need to move to tape, the expected compression ratio, etc. That being said, I do agree they market it too much and it gets confusing, as it should only be a reference for implementation and quick calculations.
- ta1243 3y agoSurely how well it compresses depends on the material you're backing up It seems they went all in on the "compressed" value after missing the doubling of the real space in LTO6.
- sebazzz 3y agoEspecially if you backup encrypted data (which isn't unusual - to have data encrypted at rest) you will have a compression ratio of 100% or even more (metadata of the compression needs to be stored too)
- monocasa 3y agoThe drives will do encryption too so that you don't have to pick between encryption at rest and compression.
- sebazzz 3y agoFrom the SSL world we know that encrypting compressed data can lead to vulnerabilities in the encryption. Does that nog apply here? https://en.wikipedia.org/wiki/CRIME?wprov=sfti1 https://en.wikipedia.org/wiki/CRIME?wprov=sfti1
- monocasa 3y agoIt doesn't really apply as both CRIME and BREACH rely on small N on the payload size, and heavy use of repeated chosen plaintexts to iterate through the secret state space.
- lovasoa 3y agoIsn't it harder when you have to later report to management that actually, your 45TB of production data do not fit on the 50TB tape ? I suppose compression ratios vary a lot between users of these things, right ? And if you are backing up user-generated data, you don't know in advance how well it will compress.
- sumtechguy 3y agoIn this case "The latest TS1170 drive supports 50TB JF media cartridges with 150TB compressed capacity through 3:1 compression" So looks like 50TB raw if I am reading this right. For these sorts of solutions you are usually not using just 1 tape. The tape is the cheap part. The drives cost a decent amount though. So in your case if the data does not fit. You buy another tape and pop it into the existing drive and finish the backup. Also you would have some sort of rotation schedule and at least a few dozen tapes. If you are doing it right offsite and onsite rotation too. An extra tape in the mix of that is not something they would worry about. When I did this for a small org I had the weekends to do a full backup of all of our computers. Then during the week I would do incrementals. At the end of the week I would ship off the prev week to an offsite and start the next full backup. Not perfect but gave us a onsite 1 day return offsite (building gone) 1 week out. The offsite would keep about 6 months of tapes and return them about once a month.
- klodolph 3y agoYeah. For most operations, you pay for drive throughput & redundancy, and the actual storage itself is “too cheap to meter”. Except when it isn’t, but those customers know who they are.
- dylan604 3y agoThese are the same customers that only make 1 tape of the data. One facility I was at required 3 copies written by 3 different writers. These were in large robot operated units, so it was all automated by our front end. One copy stayed locally, the other 2 went to 2 different offsite locations in different parts of the country with the closest being in another town across state. Each tape had to be removed from the writer and placed into a different drive to do a read back for comparison. In the LTO4 days, we would catch lots of write errors like this. I wonder if the newer formats are any better about these errors? How I miss the old days of tar/mt commands, er, no, who am I kidding!
- Johnny555 3y ago>After having helped on a lot of tape storage implementations, I get why it's useful for quick estimates on backup solutions. Can't you just do that estimate in your head? If you know your data mostly text and has a 3:1 compression ratio, when you buy your tapes you can multiply the raw capacity by 3, if it's mostly compressed images (i.e. gif, jpg, etc), you're probably not going see much compression at all. Does going with some tape drive manufacturer's estimate really help you?
- Dylan16807 3y ago> "These tapes hold 50 TB of production data, and we have about 45 TB in production. Backups and restores will be faster, we'll have fewer tapes to manage for backups" Sure, if "production" means database. Which is sometimes sensible and sometimes extremely far off. > than it is to explain the different types of data you need to move to tape, the expected compression ratio, etc. To some extent, but you mostly just need two numbers. Database capacity and photo/video/audio capacity.
- dylan604 3y agoNot once in my days of using tape storage have I ever enabled compression, so it always seems like a waste to my specific use case of storing large files of media formats, specifically video. Unless the video was in some sort of RAW uncompressed format, which is close to never, and video in a CoDec would not benefit from this compression.
- leptons 3y agoTape compression is probably not meant for your use cases, it's likely meant for large government institutions or corporations that generate petabytes of uncompressed text data a week - so instead of needing to manage compression as a separate process, the tape drive just does it without any other processing needed. The data can go from collection/creation to long-term storage without any other steps in between. For some organizations that's a big win. For the average home user backing up their plex, it might seem unnecessary.
- dylan604 3y agoLots of home users backing up their Plex with an LTO5 or greater are there?
- leptons 3y agoI am one of those home users with an LTO5, but I do not use tape compression and I have no illusions about what it was made for. I write tapes without any compression, because the stuff I write can't be compressed. Tape compression wasn't made for my use case, it was made for organizations that generate petabytes of text data.
- dredmorbius 3y agoThere's still a fair bit of mainframe-format data (zoned decimal, etc.) out there. These are typically in defined-data formats, a/k/a hierarchical , which is sort of a hybrid between data files and text files ("records" are lines, but often with multiple formats within a single file). These can achieve compression ratios ~= 90%. Standard written text typically runs about 50%. Compressed audio, image, and video data of course compress very little, if any.
- tux3 3y agoIt takes advantage of shallow purchasing decisions. If all your competitors report 30 TB*, and you report 15 TB, you may have a better product, but what the people with purchasing power want is bigger numbers, and you advertise a lower number *with 3:1 compression
- osigurdson 3y agoIf so, why not state 100:1 compression on the canonical FF*N data set.
- tux3 3y agoBecause contrary to popular belief, people are only finitely foolish But in fact this did happen to some extent. LTO tapes used to advertise with 2:1 compression ratio. Then that became 2.5:1. And now IBM is doing 3:1 And that's not because we just discovered how to improve on Lempel-Ziv. The switch to 2.5:1 was around 2012. LZMA is 20th century tech, and people still use gzip today. Compression technology doesn't account for the advertising number inflation there.
- bombcar 3y agoBecause all the companies have to do it in lockstep (see "reach vs standout" recently in tape measures).
- antientropic 3y agoBecause nobody has yet bothered to sue them for false advertising, so they continue to get away with it.
- Johnny555 3y agoBack when I still managed an LTO tape library (which was way back when LTO-4 was state of the art), our data was encrypted before writing it, also negating any compression. And back then the compression ratio was advertised as 2:1, not 3:1.
- VectorLock 3y agoThis was before the introduction of the much lauded "Middle Out" compression.
- hosteur 3y ago> The new tape cartridge media, also called 3592 70F, uses Strontium Ferrite particle technology. If anyone else is wondering if this is radioactive, it likely isn’t. > While natural strontium (which is mostly the isotope strontium-88) is stable, the synthetic strontium-90 is radioactive and is one of the most dangerous components of nuclear fallout, as strontium is absorbed by the body in a similar manner to calcium. Natural stable strontium, on the other hand, is not hazardous to health.
- PaulHoule 3y agoIn fact, small amounts of strontium seem to be good for your bones https://www.webmd.com/osteoporosis/strontium-treatment-osteoporosis https://www.webmd.com/osteoporosis/strontium-treatment-osteo...
- chaps 3y agoErm, that webmd article you linked is for strontium, not strontium ferrite -- Sr vs SrO6Fe2O3.
- BenjiWiebe 3y agoNot saying strontium ferrite is good for you, but remember that when we talk about sodium intake or iron intake or whatever, that includes multiple different forms of those, not including the pure metal. That article actually mentions two different forms of strontium and hints at others.
- chaps 3y agoThat's fair and I was curious if the same thing would happen here. But this definitely isn't my domain :) In any case, here's a lovely FDA safety data sheet of strontium ferrite. It... uh.. apparently doesn't cause pain when rubbed into the eyes of rabbits and guinea pigs. No LD50 tests were taken for the sheet though, so that's bittersweet I guess. https://www.advancedenergy.com/globalassets/non-resource-library-assets/sds-documents/strontium-ferrite.pdf https://www.advancedenergy.com/globalassets/non-resource-lib...
- Thev00d00 3y ago400MBps transfer speed means it would take 1 day, 14hours to fill it...
- izzydata 3y agoThat seems pretty good honestly for its intended use case.
- anfractuosity 3y agoYeah that seems surprisingly good as I thought the write speeds I get from a spinning disk is around 100MB/s or less.
- helf 3y ago100MB/s or less? Are you using SMR 5400rpm disks? I routinely get 180-220MB/s throughput per Exos drive.
- anfractuosity 3y agoI just ran - dd if=/dev/zero of=tmp.img bs=1G count=1 oflag=dsync And got - 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.49586 s, 113 MB/s The drive appears to be - WDC WD160EDGZ-11B2DA0, it's a 16TB disk, which I thought would be SMR, due to its size, but maybe I'm wrong. Just found https://www.reddit.com/r/DataHoarder/comments/qf5ar3/is_this_white_label_hd_inside_my_16tb_elements/ https://www.reddit.com/r/DataHoarder/comments/qf5ar3/is_this... which seems to indicate it's CMR.
- kiririn 3y ago> oflag=dsync This will reduce performance
- nayuki 3y agoThat's in the same ballpark as HDDs. It takes me ~24 hours to fill an 18 TB HDD.
- jonatron 3y agoAt what scale does tape begin to make financial sense?
- dd82 3y agowhen you need long term stable storage without getting hit by bit rot
- aspyct 3y agoPretty sure it's not immune to bit rot. Have any data to back your claim?
- perbu 3y agoTape drives will, when writing, re-read the data as it is written to make sure no errors are written and will transparently correct any errors. In addition, quite a bit of error correcting code is written, making it possible to correct simple errors. So perhaps not 100% immune, but partial immunity to bit-rot is a pretty important part of the LTO spec.
- rout39574 3y agoImmune, no. Far less prone to bit rot? yeah. My source is running backups for a significant fraction of a large univiersity for ~20 years. We had a routine cadence of tape errors, which were much less frequent than our routine cadence of disk replacement. I suppose that's anecdote. :) But it's what I've got.
- throw0101a 3y ago> Have any data to back your claim? Search for "year" in the specification of HPE's Ultrium LTO media: * https://www.hpe.com/psnow/doc/c04154430 https://www.hpe.com/psnow/doc/c04154430 Or IBM's white paper: * https://www.ibm.com/downloads/cas/Z3RYV0BR https://www.ibm.com/downloads/cas/Z3RYV0BR
- guerby 3y ago
- xyst 3y agoTIL tape drives still exist.
- 6LLvveMx2koXfwn 3y agoWe backup 1.5 PB of data per month to tape; What alternative solution would you suggest given 'cloud' in all its guises is too expensive?
- jonatron 3y agoHow much is 1.5PB of tapes, and 1.5PB of hard drives?
- jpc0 3y agoI would hazard a guess that storing tapes takes up significantly less physical space than hard drives. I don't think hard drive based storage and tape bases storage have quite the same use case.
- dagw 3y agoI would hazard a guess that storing tapes takes up significantly less physical space than hard drives. An 18TB tape is basically the same size as an 18TB 3.5" hard drive.
- adrian_b 3y agoI doubt that this can be true. I am using older LTO-7 cartridges, with a raw capacity of 6 TB. The newer LTO cartridges must be compatible mechanically with the old cartridges, because the same drive must be able to read and write them. The LTO cartridges are smaller than 3.5" drives, they have maybe about 2/3 of the volume of a 3.5" HDD. Moreover, a tape cartridge is many times lighter than a HDD. I have suitcases with 20 tape cartridges that are as convenient to carry as any typical suitcase. A luggage carrying 20 HDDs would have a very noticeable weight, and its volume would be great not only due to the bigger HDDs, but it would require serious padding to avoid HDD destruction if the case were dropped accidentally.
- laurensr 3y agoI'm still looking for an affordable external tape drive. There's RDX but their tapes seem a lot more expensive per TB. e.g. LTO 3TB costs 30 EUR RDX 2TB costs 265 EUR
- svennek 3y agoRDX is diskbased, not tape based (that is the D in the name).. https://buy.hpe.com/us/en/storage/disk-storage-systems/removable-disk-backup/c/3755725 https://buy.hpe.com/us/en/storage/disk-storage-systems/remov...
- deleted 3y ago[deleted]
- Hamuko 3y agoI've considered getting an external Blu-ray writer and just a bunch of quality inorganic blank discs as a tape alternative archiving solution.
- layer8 3y agoYou might want to consider M-Discs.
- Hamuko 3y agoAFAIK, M-Discs don't really make a that much of a difference if you're not dealing with DVDs.
- a20eac1d 3y agoDo you mean in comparison to regular BDXL Blurays?
- Hamuko 3y agoBasically all non-LTH BDs. The way I've understood it, the big selling point in M-Disc DVDs vs. regular DVDs was the inorganic dye used in it. But if you buy a non-LTH BD, you'll have an inorganic dye anyways. The company that actually made M-Discs also went bust and there's a lot of conflicting information on the Internet whether or not the current M-Discs are real M-Discs or not.
- narrator 3y agoIt's weird how tape size just seems to increase steadilyin a smooth curve to roughly match whatever hard drives are doing. It makes me wonder sometimes if they had this tech all along, but it was better just to slow roll it out so they could sell multiple generations of hardware. I guess only the material scientists at these companies know what's really going on.
- rout39574 3y agoExactly. IBM's had several generations of tape tech in the can, and has managed the rollouts with calm competence for at least back to '97 when I got started in the backup business. Their research is a prominent part of the LTO platform, too. My observation of that landscape was that they gave toys to LTO at a rate calculated to extinguish the utility of tape research in other places. This means there's a plausible competition, which IBM wins all the time. This may sound like I'm down on them for it; I'm not. Their hardware was rock solid, performed as advertised.
- MichaelZuo 3y agoHuh that's pretty smart if true.
- peppermint_gum 3y agoWell, yes, the article mentions the potential of the strontium ferrite, with existing prototypes having hundreds of terabytes of capacity. Similarly, HAMR and MAMR will be milked for years in the world of magnetic hard drives.
- vGPU 3y ago> Fujifilm has demonstrated a 580TB raw capacity tape using Strontium Ferrite particles so there is a lot of runway for future capacity increases. I would guess so.
- skywal_l 3y agoLook at Wikipedia [0], they have it planned for the next ten years like clockwork. [0] https://en.wikipedia.org/wiki/Linear_Tape-Open#Generations https://en.wikipedia.org/wiki/Linear_Tape-Open#Generations
- NoMoreNicksLeft 3y agoWow. Prosumer storage technology has rocketed past even enterprise backup technology.
- RobotToaster 3y agoKinda wish they would pack this technology into a consumer version in Compact Cassette format.
- myself248 3y agoI wonder what the world would look like if the Pereos tape drive had caught on and then been steadily improved. It was doing 1GB/tape (500MB/side) in 1994, on a tape that perches cozily on a fingertip, and theoretically should've been easy enough to build into all sorts of consumer electronics. Sony made digital cameras that recorded to floppy disk (the Digital Magnetic Video Camera, or MaViCa), and Agfa made one that recorded to PocketZip, the 40MB shrink of Zip disks. Both random-access media, but that isn't crucial for a digital camera. Could easily record to linear media, and in fact a later Digital Mavica did just that, onto CD-R discs. So recording to tape wouldn't be a stretch at all. Instead of deleting photos on the camera, you could perhaps mark them for deletion but then have that take effect at offload time. Then blank the tape and use it again. At a time when 64MB flash cards were barely affordable, a 500MB tape would've been a gamechanger. I use cameras as an example because they're probably the use-case that benefits most from a capacious writable medium in a portable format.
- g105b 3y agoSo the entire Spotify library can be stored on a single tape! What interesting times we live in.
- pdntspa 3y agoHow rich will I need to be to get one of these for my home storage array? It is hard to find a cloud backup provider that will take 10+TB without freaking out on price.
- redundantly 3y agoWithout knowing how much storage space you have nor what kind of array you own I can tell you with 99.99% certainty it would be by far much cheaper to just buy the same storage hardware and just copy your data to it.
- pdntspa 3y agoI get that but I would really like tape backup if it was viable, it's simpler and more compact.
- redundantly 3y agoTape backup isn't simple. Steer clear of it. It would be faster, easier, and cheaper to backup to a second set of disks and store them at a relative's or friend's place. Some people keep a copy of their data at their place of work. Personally I'm opposed to that.
- FullyFunctional 3y agoWhy are you opposed to that? I store an encrypted ZFS volume at work. I dont expect to ever need it but it’s just one of many redundant copy stored around.
- deleted 3y ago[deleted]
- juancn 3y ago> The TS1170 drive operates at 400MBps raw throughput Am I reading this right? At that throughput it would take about a year to fill a compressed tape (assuming 3:1 ratio is achievable). Or is there some parallelism involved? (i.e. many tracks written in parallel to the tape, I don't know a lot about tape drives)
- mgerdts 3y agoWith 3x compression and 900 MB/s it only takes 46 hours to fill one. But maybe I’m bad at math. Perhaps more importantly it will take that long to do a full recovery from one.
- chrisco255 3y agoThat's nearly 1 GB/s. ~18 minutes per terabyte. What would you be doing a full recovery of on 150 terabytes of data? This type of storage is usually for archive data not for quick restoration of active servers or databases.
- klabb3 3y agoBtw what does 3x compression mean in this case? Does the device itself do compression and if so how can it be a fixed rate, given the data is arbitrary?
- stonogo 3y agoYes, the compression is on-device, and it's not a fixed rate, it's a peak. Efficiently-stored data like video compresses less well or not at all.
- tempay 3y agoLTO drives include compression and manufactures quota both the guaranteed/native capacity/speed as well as the "compressed" capacity/speed. In they case of IBM they always use a 3:1 ratio in their specs. Of course in reality it depends on the data you feed it so the ratio has always seemed odd to me but it is what it is.
- exabrial 3y agoI wish the tape hardware itself wasn't so pricey! would love to have local backups our our colo servers.. alas we pipe everything to "cloud" storage (remembering that the cloud is just someone else's server in Virgina)
- leetrout 3y agoI am so tired of the "someone else's server" trope as if there is no value add. S3 is a fantastic piece of architecture and service and the fact that the common dev has such tools and service available instantly from their personal laptop or even phone is mind boggling. We are very fortunate that AWS led more services being made available to common folk than what had been done in the past. Its not all about cost or hardware ownership--the value these near infinitely scalable services is easy to ignore but astounding at the grass roots level.
- anonym29 3y agoAs someone who constantly scrapes and indexes every bit in every public S3 bucket I can find, for purposes of selling query access to my indexes, I couldn't agree more - FANTASTIC value add. Of course, I never have, don't now, and never would run a business on someone else's computer though... all the hyperscalers will happily charge you more for 3 months worth of compute than the cost of the hardware the compute is being done on. Would I rather build a server for $5k or pay $1600/mo to rent that exact same server? Such a difficult decision! /s
- cebert 3y agoDo you feel like that practice is unethical at all?
- johnnyworker 3y agoProbing for open buckets and selling the results for people likely looking to do nasty things with the is unethical IMO, but it's still more ethical to do it and then bring it up to warn others of it, than to just do it. I mean, how do you know if the people talking about this fantastic value add at the fingertips of anyone are not doing it? When it comes to doing basic napkin math and publish the results, that I find highly ethical, actually.
- dtjohnnymonkey 3y agoAs a kid I asked my dad to buy a tape backup drive so I could do backups of our Win95 machine. I’d run backups religiously, to the chagrin of other members of the household who were unable to use the machine during a backup. Later at my first job I was responsible for performing tape backups and rotating them. The most recent would go into the fire safe, and the one before that would go with the owner to take home.
- simonblack 3y agoThe problem with tape drives for backups is that they only just outpace the sizes of hard drives over the years and are multiples of hard drive prices. Most of us have used tape drives in the past for backups only to find that hard drives work better, are more reliable overall, can be accessed randomly, and are cheaper. Imagine using a 2Gig DAT tape for backups these days, as I did do way back in the past. Or a 60MB QIC cartridge, as I did do a few years earlier than that DAT tape drive.