6 ms·
Does it mean LTO tape for the win then? We’re about to start a project to build an LTO-9 based in-house backup system. Any suggestions for DIY Linux based oper
by alchemist1e9 2y ago
Does it mean LTO tape for the win then?
We’re about to start a project to build an LTO-9 based in-house backup system. Any suggestions for DIY Linux based operation doing it “correctly” would be appreciated. Preliminary planning is to have one drive system on in our primary data center and another offsite at an office center where tapes are verified before storage in locked fireproof storage cabinet. Tips on good small business suppliers and gear models would be great help.
- antisthenes 2y agoWhat's your budget?
- alchemist1e9 2y ago$50K if needed but it doesn’t look to need that. 2PB initial data. predicted 1PB/year with around 10-30%/year rate of growth of rate of growth (acceleration?)
- akira2501 2y ago> fireproof storage cabinet. Nothing is fire proof. Is the cabinet "fire suppression system liquid" proof? > Tips on good small business suppliers and gear models would be great help. Hire an auditor would be my advice. Every business is different. I am, just now, having flashbacks of when I was in a SOX environment and had to regularly contract with them... and while the experience can be somewhat unpleasant I've often found good auditors to be extremely knowledgeable about solutions and their practical implementation considerations.
- hinkley 2y agoPretty sure the pyramids at Giza count as fireproof boxes. But that’s out of most people’s price range.
- burnished 2y agoNot if the fire is big enough.
- hinkley 2y agoWhen a catastrophic event is catastrophic enough, your other problems cease to be your problem.
- akira2501 2y agoThey're not fully sealed. There are two shafts which connect the King's chamber to the exterior of the pyramid. The lower ritual congregation area is not fully sealed off from the upper chambers either. Which means bats are a constant problem in pyramids. You could easily get a fire going inside one.
- dredmorbius 2y agoFortunately, it actually is somewhat challenging to get a fire going inside a bat.
- akira2501 2y agoFamously, their feces are not at all challenging to set on fire.
- dredmorbius 2y agoStill on the outside of the bats ;-)
- hinkley 2y agoThe advice I got long ago from an IT guy was: if you wait long enough, tape will be on top again. That was a long time ago but I’ve peeked in at backup systems in the intervening years and it does seem to hold true over time. But it really depends how much data you have. My ex dropped a single HDD in a safety deposit box at CoB, N times per week and fetched back the oldest disk. I don’t think she ever said how many were in there but I doubt it was more than three. I think the CTO took one home with him once per week. The silly thing about most of this set up is that the office, the bank, and the data center were all within half a kilometer of each other. If something bad happened to that part of town they only had the infrequent offsite backup.
- katbyte 2y ago20tb and under is easy. 500tb is hard
- exe34 2y agothe sort of bad that would render all three unreadable would probably melt the rest of the infrastructure the company relies on for business anyway.
- marcosdumay 2y agoEvery time I've looked, tapes were on top "again" for large scale archival. And I've been looking for ~20 years by now. I don't get where people get the impression that X was at the top right before tapes got that last innovation (where X here is most often HDDs, but not always). But that's always the impression, and tapes are always on top. People also have been working with 3D phase change drives since the 90s. Those always promise to replace tapes. But nobody ever got them robust enough to leave the labs.
- hakfoo 2y agoTape might win for large-scale systems, but it's basically dead for home-office scale. You used to be able to get modestly priced tape units for home use from the old "connects to the floppy controller" units with capacities in the tens of megabytes, up to some late-gen SCSI/IDE/parallel/early USB models that would be a couple of gigabytes, but still at home-friendly prices. What's today's answer? An enterprise-grade device that might put 10TB+ on a tape, but comes with a four-figure price tag and isn't really sold at Best Buy. If I want to back up the house today (maybe 4 active PCs, 5-6Tb of total space), affordable choices are pretty much disc-based. I could choose a cheap NAS (ended up doing that with an old fanless Atom machine and a used 12TB datacentre drive) or get a USB-attached external drive. Even if I used BD-XL media, even my modest needs would be dozens of discs, plus getting a writer in a shrinking market. There are plenty of datahoarders with much bigger needs, but even for them, tape is completely outside the addressable market.
- WaitWaitWha 2y agoQ: Why not archival M-Disc?
- actionfromafar 2y agoA: Are there any "real" M-Discs for purchase anymore? (I.e. not just rebranded regular dye-discs.)
- netrap 2y agoPossibly in DVDR.
- alchemist1e9 2y agoAmount of data makes it less realistic. We have around 2PB data currently and expect to grow around 1PB next year with maybe 10-30% annual growth rate.
- bluedino 2y agoTapes are fun. You can fit a petabyte of data in a bankers box! The problem quickly becomes: - Do we have a drive that can read this tape? - Do we have server we can connect it to? - Do we have storage we can extract it to? (go ask your internal IT team for 10TB of drivespace...) - What program did we create this tape with? Backup Exec, Veritas, ArcServe, SureStore - You have the encryption keys, right? - How much of this data already exists on the previous months backup? - Who's going to pay for the storage to move it to Glacier/etc? -How long is it going to take to upload?
- aftbit 2y agoIf you are having trouble getting 10TB of disk space from IT, you have bad IT. Not saying that's uncommon or anything, but 10TB fits on one external hard drive for $300 from Best Buy, or less than $1000/mo using EBS on AWS if you need some better guarantees and are all-in on the cloud.
- DaiPlusPlus 2y ago> Tapes are fun. You can fit a petabyte of data in a bankers box! Yes, though those ultra-thin M.2 NVMe drives could probably top that now. > Do we have a drive that can read this tape? Don't let this be a problem in the first place: buy 4 tape drives and keep 2 of them in your cold/offsite/airgapped storage site (2 in case 1 fails, so you can use the remaining drive to transfer everything to a newer format). > Do we have server we can connect it to? Significant hardware should not (and is not) necessary: the LTO-7/8/9 drives I see for sale right now seem to be using either USB 3.0, Thunderbolt, or SAS connections; USB and Thunderbolt can be handled by any computer you can find at a PC recycler today; while any old desktop can handle SAS with a $80 HBA card. > Do we have storage we can extract it to? (go ask your internal IT team for 10TB of drivespace...) 10TB isn't a good example (case-in-point: I have a 3-year-old stack of unused 12TB WD drives less than 3 feet away from me). That said, if you're enterprise-y enough for a $4000 LTO-9 drive, then you probably also have a SAN that's chock full of drives, so being able to provision a 10TB+ LUN should be implicit. > What program did we create this tape with? Backup Exec, Veritas, ArcServe, SureStore Ideally, none of those; instead, good ol' Perl and `dd`. > You have the encryption keys, right? I don't encrypt my backups to avoid this problem. My old data archives have little exploitable value for any potential attacker; and I imagine I'd store backup tapes this important in a fireproof safe in my parents' house or something. I'd only encrypt the entire tape if the tape were to leave my custody. I appreciate that this is not for everyone, and it's probably illegal for some people/orgs to not encrypt backups too anyway (HIPAA, etc). > How much of this data already exists on the previous months backup? Incremental/Differential backups are still a thing. > Who's going to pay for the storage to move it to Glacier/etc? No-one should. Cold data backups should/must always be in the custody of a designated responsible officer of the company. BUUUT, I guess there's nothing wrong with storing an encrypted copy in Cloud storage (as in S3/Glacier/AzureBlobs - not OneDrive...). I actually do this right now thanks to the smooth and painless integration in my Synology NAS. It costs me about $15/mo to store all these TBs in S3. > How long is it going to take to upload? Consider that it's 2024 - a company with LTO-9 and SAN is probably going to have a metro-ethernet IP connection at 10Gbps or even faster. At home I have a 10Gbps symmetric connection from Ziply (it's $300/mo and they give you an SFP module, which I put into my Ubiquiti UDM): so the limiting factor here is not my upload speed, but my drive read speed (LTO-9 drives seem to read at about 2-3Gbps raw/uncompressed?)
- gosub100 2y agoMake sure the bandwidth exists to keep up with the write speed of the LTO drive. For instance, the write speed for LTO-6 (which I own as a hobbyist) is around 300MB/s, but line speed of gigabit Ethernet is about 100MB/s. Translate those numbers to LTO-9 and make sure that the NAS, network, or local storage can keep up. It's not a deal-breaker to underflow the drive, but it causes the tape to stop, rewind, and re-buffer (called shoe-shining) which takes more time and causes unnecessary wear on the drive and cartridges.
- adrian_b 2y agoLTO-9 tapes can be easily found on Amazon in many countries, made by IBM, HP, Quantum or Fuji. The vendor does not matter, whichever happens to be cheaper at the moment is fine. For the tape drives, the internal drives can be cheaper by around 10%, but I prefer the tabletop drives, because they are less prone to accumulate dust, especially if you switch them on only when doing a backup or a retrieval. The tape drives have usually very noisy fans, because they are expected to be used in isolated server rooms. I believe that the cheapest tape drives from a reputable manufacturer are those from Quantum. I have been using a Quantum LTO-7 tape drive for about 7 or 8 years and I have been content with it. Looking now at the prices, it should be possible to find a tabletop LTO-9 drive for no more than $5000. Unfortunately, the prices for tape drives have been increasing. When I have bought an LTO-7 tabletop drive many years ago it was only slightly more than $3000. The tapes are much cheaper and much more reliable than hard disks, but because of the very expensive tape drive you need to store a few hundred TB to begin to save money over hard disks. You should normally make at least two copies of any tape that is intended for long-term archiving (to be stored in different places), which will shorten the time until reaching the threshold of breaking even with HDDs. Even if there are applications that simulate the existence of a file system on a tape, which can be used even by a naive user to just copy files on a tape, like copying files between disks, they are quite slow and inefficient in comparison to just using raw tape commands with the traditional UNIX utility "mt". It is possible to write some very simple scripts that use "mt" and which allow the appending of a number of files to a tape or the reading of a number of consecutive files from a tape, starting from the nth file since the beginning of a tape. So if you are using only raw "mt" commands, you can identify the archived files only by their ordinal number since the beginning of the tape. This is enough for me, because I prepare the files for backup by copying them in some directory, making an index of that directory, then compressing it and encrypting it. I send to the tape only encrypted and compressed archive files, so I disable the internal compression of the tape drive, which would be useless. I store the information about the content of the archives stored on tapes (which includes all relevant file metadata for each file contained in the compressed archives, including file name, path name, file length, modification time, a hash of the file content) in a database. Whenever I need archived data, I search the database, to determine that it can be found, for instance in tape 63, file 102. Then I can insert the corresponding cartridge in the drive and I give the command to retrieve file 102. I consider much better the utility "mt" of FreeBSD than that of Linux. The Linux magnetic drive utilities have seen little maintenance for many years. Because of that, when I make backups or retrievals they go to a server that runs FreeBSD, on which the SAS HBA card is installed. When a tabletop drive is used, the SAS HBA card must have external SAS connectors, to allow the use of an appropriate cable. I actually reboot that server into FreeBSD for doing backups or retrievals, which is easy because I boot it from Ethernet with PXE, so I can select remotely what OS to be booted. One could also use a FreeBSD VM on a Linux server, with pass-through of the SAS HBA card, but I have not tried to do this. My servers are connected with 10 Gb/s Ethernet links, which does not differ much from the SAS speed, so they do not slow much the backup/retrieval speed. I transfer the archive files with rsync over ssh. On slow computers and internal networks one can use rsync without ssh. I give the commands for the tape drive from the computer that is backed up, as one line commands executed remotely by ssh. The archive that is transferred is stored in a RAMdisk before being written on the tape, to ensure that the tape is written at the maximum speed. I write to the tape archive files that have usually a size of up to about 60 GB (I split any files bigger than that; e.g. there are BluRay movies of up to 100 GB). The server has a memory of 128 GB, so I can configure on it a RAMDdisk of up to 80 GB without problems. This method can be used even with a slow 1 Gb/s or 2.5 Gb/s network, but then uploading a file through Ethernet would take much more time than writing or reading the tape. There is one weird feature of the raw "mt" commands, which is poorly documented, so it took me some time to discover it, during which I have wasted some tape space. When you append files to a partially written tape, you first give a command to go to the end of the written part of the tape. However, you must not start writing, because the head is not positioned correctly. You must go 2 file marks backwards, then 1 file mark forwards. Only then is the head positioned correctly and you can write the next archived file. Otherwise there would be 1 empty file intercalated at each point where you have finished appending a number of files and then you have rewound the tape and then you have appended again other files at the end.
- wazoox 2y agoArchival is more of a process than only a question of media. First you must create a proper database of your archived data. Maybe you want to do 3 copies, not two. Maybe you want to use two different archival formats such as tar and LTFS, just in case. Maybe you want to source your media from both available producers (Sony and Fuji) because in the long run, maybe one or the other may grow some funky error mode or corruption problem. Etc. Also check my tape management primer: https://blogs.intellique.com/tech/2022/01/27#TapeCLI https://blogs.intellique.com/tech/2022/01/27#TapeCLI