2 ms·
LTFS (Linear Tape File System) shows each each "file" as often a pretty dumb "write-only" mechanism / append-only file system. That is: if a file A is modified
by dragontamer 5y ago
LTFS (Linear Tape File System) shows each each "file" as often a pretty dumb "write-only" mechanism / append-only file system.
That is: if a file A is modified, LTFS "represents" this by marking fileA as dead, then copying most of the contents of A to the end of the tape... with the few modifications in the correct location.
Because tapes have so much capacity, this ends up working much better in practice than you'd expect. Especially if your backup-like software (or Apache-log files or whatever you're archiving) are largely append-only (ex: Wikipedia dumps or whatever huge data-storage requirements you're doing).
-----------
Allegedly anyway, I don't really use tapes myself, but I amuse myself by looking at their specs / usability every now and then. It seems like the details of LTFS depend on each manufacturer's system, but it seems like LTFS itself is somewhat cross-compatible.
Some LTFS-library docs suggest that different tapes are different folders. Others suggest that different tapes are mount/unmounted (which sends a sequence of commands to the little tape-robot to fetch the correct tape, which it does so by scanning barcodes or maybe going to correct slot)
-----------
Tapes are meant to be used in a "jukebox-style" application IMO. If you need just one tape, you're doing it wrong. If you need 80-tapes, then doing it manually is obviously wrong, you should pay a bit extra for that "jukebox / tape-library" robot to fetch the right tapes automatically.
> Does it already exist as open-source?
Red Hat claims LTFS support. But I imagine this is only "disk-level LTFS", and not "library / robot LTFS".
- metadat 5y agoForgive my ignorance, does LTFS include the ability to use a pool of spinning rust or SSDs in front of it? That part sounded fancy and cool.
- dragontamer 5y agoOh, that sort of thing would be a tape-NAS sort of appliance and costs big $$$$. I know IBM talked about it before. I think here's their PDF of some university that did this: https://irp-cdn.multiscreensite.com/b076c94c/files/uploaded/FLAPE_IBM_EDGE_Presentation_FINAL.pdf https://irp-cdn.multiscreensite.com/b076c94c/files/uploaded/... --------------- The distributed data-thing is pretty common in the supercomputer world. I know that the nodes on the "Summit" supercomputer each have a local SSD for caching, and a centralized hard-drive array for longer-term storage. I'm pretty sure the software just manually copies the data around as necessary (that is: any software written for the supercomputer can copy data from the central storage into a local SSD). Supercomputers can have that kind of simplicity, because you buy 3000+ nodes with the same specs. So you know all nodes will have that 1.6TB SSD available. No matter which node your software ends up running on, the SSD location is set and obvious. It doesn't matter how the data is archived: you read it in and wait (it could be tape, hard drives, or any other mechanism)