10 ms·
That's good. But when will it ever move out of the enterprise market? What I really want is an affordable Tape storage for us common folks / home users to arch
by webmobdev 6y ago
That's good. But when will it ever move out of the enterprise market?
What I really want is an affordable Tape storage for us common folks / home users to archive data long term. It's really a pain to keep transferring your old backups every 2-5 years to a new CD / DVD / portable HDD. There is a market for home users that really don't want to put their data on the "cloud". And this generation is really creating so much data, some of which, I am sure they would like to store long-term.
- chrisseaton 6y agoI store on an external HD, and every year I buy a new HD and copy across to it and verify checksums against my records. Costs basically zero and low-effort. What am I missing?
- sgt 6y agoEvery year you buy a new HD and yet the cost is basically zero?
- chrisseaton 6y agoYeah what does a new HD cost? Like £100 max for a big high quality one? Compared to buying a £5k tape drive and expensive tapes... yeah that's basically zero.
- jjulius 6y ago$100 is definitely not "basically zero". Not only that, but a $5k tape drive and tapes are designed to last a significantly longer period of time, essentially bringing the cost of long-term storage closer to $0/year than your option of spending $100/year.
- aidenn0 6y agoIt's 50 years before $100/year catches up with a one-time cost of $5k. On top of that, the total capacity of the mediumgoes up steadily over time, while you will need to drop another $5k every 10 years or so or span your backups across multiple tapes.
- sgt 6y agoIn reality I am sure he could get away with buying a new drive every 5 years.
- stratosgear 6y ago5k amortized at 100/year is 50 years to get your money's worth. It seems 100/year is a much better deal. I don't see your math working out...
- webmobdev 6y agoThat's assuming a consumer tape drive will cost that much. They'll succeed only if they cost less than $500. Enterprisey stuffs are always costly. Just like consumer grade HDDs vs entrprise HDDs.
- chrisseaton 6y ago> a $5k tape drive and tapes are designed to last a significantly longer period of time You're going to use the drive for 50 years? 50 years? Half a century? And you think it'll be working and supporting a format with a useful capacity in half a century?
- webmobdev 6y agoYou miss the point - consumer tape drives won't cost 5k, just like consumer HDD's are cheaper than enterprise HDD's.
- sgt 6y agoAbout consumer tape drive reliability though...
- centimeter 6y agoWhy do you do this manual checksumming thing instead of e.g. setting up a ZFS mirror once every n years?
- chrisseaton 6y agoBecause simple and manually inspectable is better for backups.
- centimeter 6y agoWhat do you do if you find a checksum error? ZFS mirror will fix it for you.
- chrisseaton 6y agoWhat do you think I do? Copy from the last good copy.
- dragontamer 6y agoZFS does that, except with error correction codes applied against bitrot. All you gotta do is turn on a ZFS-system and hit the "scrub" command. That automatically checks all checksums: and error-corrects all data that fails a checksum. If anything, ZFS makes that whole system you just described easier and more automatic. Its not really hard for me to push the "scrub" button on my Nas4Free box every few months.
- pishpash 6y agoWhat do you do with errors within the latest increment? A shelf of annual HDD's doesn't sound too bad otherwise.
- jjulius 6y ago>... every year I buy a new HD... OK, but OP said: >It's really a pain to keep transferring your old backups every 2-5 years...
- chrisseaton 6y agoBecause they're considering options like DVDs. With a new HD you can just copy it across in one step.
- jjulius 6y agoOP explicitly mentioned HDDs in their post; they do not want to be transferring data to a new HDD every year, or two years, or five years.
- chrisseaton 6y agoWell that's why I asked what I'm missing! It's just one command to copy from an old HD to the new HD? Takes me all of 30 mins once a year.
- webmobdev 6y agoYou have to save your important data in multiple mediums to keep it safe. This is in case one medium fails. Obviously, cost being a factor, optical disks is one of the mediums to be considered despite all its other downsides (slow transfer speed).
- tomcam 6y agoNot trying to be rude or sarcastic here: what is that command? Certainly in my experience on both macOS and windows, the default operating system GUI mass copy invariably poops out halfway through, with no explanation as to why. Maybe something like this? rsync --archive --acls --xattrs --checksum --vvv /src /dest
- pizza234 6y agoThe OP is mixing mediums with different characteristics. The DVD has a fixed size (say, 5.x GB), while hard disks are relatively open-ended. One can buy 10+ TB magnetic disks for a cheap price (less than 200$). If the OP really needs dozens of TBs of capacity every few year, they definitely don't fit in the home user market they are talking about.
- weare138 6y agoLow cost, high density and a low failure rate compared to HDs. That's why it's still used in the enterprise for backup solutions.
- chrisseaton 6y agoI don't think these are home-use concerns.
- martyvis 6y agoActually nearly every enterprise backup system my company has deployed in the last 5 years is tapeless. Most systems involve snapshot backup management that are replicated across multiple disc/SSD systems. (I'm the network guy, so not my focus area)
- KingMachiavelli 6y agoTape is moving from just 'enterprise' to cloud provider enterprise. The periodic cost of having a tape storage/management system installed is a hard pill to swallow. Most companies still use tape but they just access it via AWS Glacier.
- Tokkemon 6y agoNormal people don't know what a checksum is.... or they think that's the amount their paid every week from the bank.
- chrisseaton 6y agoThe checksum isn't the relevant bit. That doesn't change whether you use a hard drive, a tape, or a DVD. Forget about it as part of the discussion if you want.
- webmobdev 6y ago> What am I missing? Money - The cost factor of buying the new HDD. And to be safe, you need to have more than one medium. That's again more cost. Time - Ensuring that your existing machine is compatible with both your old and new disks (IDE vs SATA or working DVD drives for example).
- agumonkey 6y agoLet's have someone invents a ultra high def head to repurpose all the old VCRs and VHS tapes :)
- jdofaz 6y agoD-VHS was a thing, tape capacity started at 25GB
- Tsiklon 6y agoThat would be wonderful, unfortunately the storage density of tape is dependent also on the magnetic particle in the tape; smaller particles which can be magnetised enable smaller heads, which in turn enable higher storage density. I know your comment was a throwaway idea but I got a little bit too involved looking up information around this answer - so I've committed further and looked a little deeper to avoid looking too stupid: Currently with BaFe (Barium Ferrite) as their magnetic particle, LTO-7+ tapes have particle sizes less than 100nm in size (https://indico.cern.ch/event/637013/contributions/2669089/ https://indico.cern.ch/event/637013/contributions/2669089/ - check out slides 14 and 15 in the powerpoint.). VHS on the other hand is a little more tricky to find information on I suspect due to the age of the technologies involved it's going to be significantly bigger - but the closest I've found to a technical document is a student paper at NYU which refers to this IEEE Paper (https://ieeexplore.ieee.org/document/50474 https://ieeexplore.ieee.org/document/50474) saying that the average particle size for VHS is ~300nm and for S-VHS at ~150nm. But that first presentation also mentions smoothness increases over time with LTO, which suggests improvements in the coating process. I suspect improvements in the coating, combined with both the size of magnetic particle shrinking, enabled an increase in density of particles on the tape. This in conjunction with improved thinness of the tapes enabled large increases in tape length for the same cartridge size (nearly double the length from LTO-1 -> LTO-8), has in turn led to the enormous jumps in capacity we've seen in LTO. Meaning unfortunately I don't think it'll be possible to do super interesting things with VHS.
- devwastaken 6y agoThe solution is generally to not be doing full duplicative backups but to check hashes between already backed up files and new files, only backing up what you need. I don't recommend backing up to HDD's. They are prone to early failure from portable use because of vibration and drops. What we need are more affordable sata SSD's. Currently nvme SSD's are very close to the same price. If you could sell a 1TB portable usb 3/usb c backup device that comes with good software the vast majority of people would be set.
- deleted 6y ago[deleted]
- wtallis 6y ago> I don't recommend backing up to HDD's. They are prone to early failure from portable use because of vibration and drops. Your backup device should stay at home, on the desk or in the closet where you hide your networking equipment. There's very little benefit to trying to carry around your backup device with your laptop on a regular basis, and if you do, you still need another backup device that isn't going to get stolen at the same time as your laptop.
- ksec 6y ago>affordable sata SSD This. Even 100MB/s SSD will do. We need a type of NAND and SSD that offers High Capacity and low speed and low cost. The Current 2TB is ~$170, compared to a 2TB Portable HDD at ~$60. And I am wondering, if Silent File Corruption, bitrot etc are things of the past on SSD? Does BTRFS / ZFS even make sense on SSD? [1] https://www.amazon.com/Green-1TB-Internal-SSD-WDS100T2G0A/dp/B07NNRTTCM?ref_=ast_sto_dp https://www.amazon.com/Green-1TB-Internal-SSD-WDS100T2G0A/dp...
- Ekaros 6y agoSSD and flash storage can have their own issues. For long term unpowered storage there might be issues with data retention. Not so much with magnetic media. And with bad drivers there is risk of ruining parts of disk by writing too many times...
- systemvoltage 6y agoAWS Glacier?
- ghaff 6y agoThe post you were replying to specifically didn't want to use "the cloud." But for single-digit TB, which covers consumers who don't either have a huge number of video files or is a serious data hoarder, the answer is the use local disks. And,if they are OK with cloud storage which they probably should be, something like Backblaze is probably a better choice.
- monocasa 6y agoSo having worked in the tape library space, they don't really make sense for home users. They say that they have a 50 year lifespan, but everyone in the industry knows that's a blatant lie these days. The tape customers transfer everything over to the new generation every 2 to 3 years, so it's never tested and none of the customers really care. You also only get a couple hundred writes to a tape in it's lifetime. That includes bulk appends as the stress from seeking is a major factor of what kills them. They really need to be treated as WORO media, bulk, the whole 12TB or what have you at once. Not a great fit for home use.
- est31 6y agoThat's precisely what you need if you want to upload some backups from NAS systems or similar.
- monocasa 6y ago$100 each starts making way less sense when they have the same semantics as a burned optical discs for home users. These days the main benefit to tape is the physical safety of having the media be inside a cartridge, and the streaming read speeds are stupid fast so you can reconstruct a backup faster. Home use, even with a data hoarder sized NAS, is better served with a bluray burner with multiple copies.
- pizza234 6y agoA 12 TB HDD requires 10 GB/day for three years to be filled. This is not home market, it's professional market or hoarding (by today's standards). Objections on the failure rate of HDDs are absolutely valid, but then one should also need to consider the bigger picture (e.g. storage loss), in which case, having a remote copy is also important.
- segfaultbuserr 6y ago> A 12 TB HDD requires 10 GB/day for three years to be filled. [...] hoarding (by today's standards). You have an interesting argument - by the same reasoning, when one has eventually filled up a 12 TB HDD, it would no longer be hoarding by tomorrow's standard. In other words, at this point, one should be able to get the next generation of HDDs for cheap, and it's this fact that makes all tape drives unnecessary. Now I wonder whether it was a mistake for me to buy a spare 14 TB HDD to 1:1 mirror my new file server for cold backup. Perhaps a smaller one should be good for 5 years anyway...
- throwaway201103 6y ago> 12 TB HDD ... This is not home market, Indeed. The home directory on my PC at home has 80GB of files stored. Most of it is just there out of laziness. About the only thing I try to keep backups of are my tax returns. What do people have on a home machine that needs enterprise-grade backups? You might say photos and videos. I guess that's a personal thing; I realized a long time ago that I never spent any time actually looking at any of the pictures I took, so I stopped taking them.
- UMetaGOMS 6y ago100% agree. I have a NAS for photos and media, plus a big USB HDD for ZFS snapshots of my machines - but all of this is homelab tinkering, not an actual backup strategy. I realized in the last year or so that the only digital media I have which I would be genuinely sad to lose were wedding photos - so I saved those to 3 different cloud providers, made a Blu-ray copy, and a copy on the NAS. If I lose all of that in some tragedy, chances are I've got bigger things to worry about.
- 6y ago
- baybal2 6y agoExcept for a brief period in nineties, tape never been aimed at consumer storage. I think the market for tape will always be the same few giant companies for whom it will be priced just a bit less than the next cheapest option.
- bluedino 6y agoHome users had tape backups since the 80’s. The problem was, nobody actually backs up so not enough people bought tapes/drives so they quit making them.
- rualca 6y agoWell, nowadays digital documents, such as videos and photos, are much more important and widespread and mundane than they were a couple of decades ago. During the 80s there was no digital consumer camera market. Nowadays every person can easily generate hundreds of megabytes of photos and videos per day. Each hour of a 4K video can be close to 7GB, and we're already seeing cellphones which are able to record 4K video at 60fps and 1080p videos at 240fps.
- webmobdev 6y agoShucks - It was too early for its time. Did they even have RAM in MB's during the 80's? :)
- chihuahua 6y agoJust barely. It was possible to buy a PC with 2 or 4MB of RAM in the late 1980s, but you couldn't do much with it. On Windows/DOS, I think applications had to be written specifically with "extended memory" or "expanded memory" in mind. There were 2 incompatible "standards" for how to organize memory beyond 1MB. On the Mac side, in 1989 you could apparently buy a Mac IIci for $6200 with 1MB or 4MB, "expandable to 128MB".
- wazoox 6y agoI built such a cloud service; you can archive the data uploaded on our cloud storage to tape on demand, and we'll send you the tape back when requested (you basically buy the tapes once, and they're yours, no string attached). It failed to gain any traction, though.
- webmobdev 6y agoSounds like a good idea. But I am guessing it failed as it wouldn't be attractive for common folks because, while the tapes might be affordable, tape drives are really expensive and uncommon.
- wazoox 6y agoIn fact the service is tentatively targeted to content producers. Most post-houses do have LTO drives (it's a requirement to work with NetFlix among other to archive to LTOs). The idea is that any content owner can bring their tapes to the post house when needed (where they would actually use it) instead of USB disk drives.
- webmobdev 6y agoHaving given some thoughts, your potential business success depends on convincing the clients to not buy a tape drive, and addressing the above concerns. Smart pricing can address the former, but the latter needs more brainstorming. Some of the issues you need to address: - Trust. How can someone trust you with potentially proprietary / copyrighted data? - Speed. Is it faster to copy the data to a tape locally, then to transfer to you? - Reliability. How can the client be sure that you have made a successful backup? - Advertising. Did you reach your target base to make them aware of your product and address all their concern to make them consider it?
- wazoox 6y agoYes, that's the point. When you're a content creator and you're making at most a couple of TB of data per month (typical figures), hardly enough to fill a handful of tapes a year, it's hard to justify the expense for a tape drive. OTOH cloud storage is either too complex (S3, Glacier) or too expensive (Dropbox, Box) for this volume of data. Trust: yep, hard to say for this one. At least I can provide actual guarantees (for instance, none of our storage is out of the country). Speed: it's possible to send us tapes directly. However the main selling point of our solution is to allow people to upload relatively small volumes of data continuously, then archive it all to tape in large batches when you have enough to fill an LTO. Reliability: the interface allows the user to see where their files are (on which tape), to run checksum controls on tapes, and restore from tape. Every tape is sold with 3 free operations a year (archive, checksum verification, restoration). Advertising: that's clearly not our strong point :)
- SoSoRoCoCo 6y agoTotally! I have a friend who does sound engineering for hollywood and he has two firesafes full of HDDs, SSDs and TAPE with hundreds and hundreds of terabytes of data. I couldn't even finish the sentence "Have you tried the cl..." before he cut me off and was like: way too much data. Didnt have the heart to tell him two of those three media types need to be powered on periodically...
- tomcam 6y agoOw! Telling him would be a kindness.
- pochamago 6y agoHDDs can't require powering all that periodically. I just found my hard drive from college, and plugging it into my computer just worked. Great finding all my old documents and music that I'd forgotten about.
- sp332 6y agoThe lubricants might settle, causing sticking of various components when you try to power it on after a long time.
- ekianjo 6y agoIf your drive uses ZFS you apparently need to power it on more regularly than that to avoid bit rot.
- hpfr 6y agoWhy would the file system affect bit rot rate?
- aaronmdjones 6y agoIt doesn't. What matters is that you have a filesystem that can detect and repair bitrot. For that to work, it needs to check everything occasionally, which means they need to be powered up occasionally. If you don't do that, eventually you'll get to a point where it can't repair anything, and then you gain nothing over using a filesystem that doesn't do this, is the point.