5 ms·
From what they explained on IRC, this is home-made hardware (not yes disclosed because of pending patent), low-level storage without a filesystem. They split th
by renchap 10y ago
From what they explained on IRC, this is home-made hardware (not yes disclosed because of pending patent), low-level storage without a filesystem. They split the"safe" in chunks, encrypt it, calculate parity blocs, and store them all on a different hardware. They intend to disclose the software "soon", and the hardware later.
- tiernano 10y agosounds similar to backblaze... though they use XFS (IIRC) for their file system, but do the RAID type stuff themselves...
- renchap 10y agoNope, with Backblaze you have instant access to an individual file. Here you need to unarchive your safe, which can take up to a few hours (depending on the size of the vault), and only then you have access to the content of the safe. No way to retrieve only 1 file, or have instant access.
- tiernano 10y agoi dont mean their service is like backblaze, i mean their underlying tech...
- rsync 10y agoWait, this isn't random access (live) storage ? How do you rsync (or borg, duplicity, whatever) to a dataset that has to be checked out ? I can imagine some ways of doing that, but am curious ...
- insulanian 10y agoIt is looks to be more like AWS Glacier, not like S3.
- rakoo 10y agoIt says on the page that there's a 7 days window during which your data is freely accessible and mutable, after which it is permanently stored in the homemade infrastructure.
- cm3 10y agoYou're right about their cross-rack RAID-like stack, but didn't Backblaze switch to ext4 in the meantime?
- tiernano 10y agopossibly... cant see anything about it on storage pod 6 [1] but i do remember mentioning a file system, i just though it was XFS... [1]: https://www.backblaze.com/blog/open-source-data-storage-server/ https://www.backblaze.com/blog/open-source-data-storage-serv...
- cm3 10y agohttps://www.backblaze.com/pod.html https://www.backblaze.com/pod.html says ext4. https://www.backblaze.com/blog/vault-cloud-storage-architecture/ https://www.backblaze.com/blog/vault-cloud-storage-architect... Each of the drives in a Vault has a standard Linux file system, ext4, on it. This is where the shards are stored. There are fancier file systems out there, but we don’t need them for Vaults. All that is needed is a way to write files to disk, and read them back. Ext4 is good at handling power failure on a single drive cleanly, without losing any files. It’s also good at storing lots of files on a single drive, and providing efficient access to them. They have since migrated to ext4, but judging by their job description they might still have JFS in use. https://www.backblaze.com/blog/seeking-data-protectors/ https://www.backblaze.com/blog/seeking-data-protectors/ Diagnose and repair file system problems with JFS and ext4 and debug other system problems https://www.backblaze.com/blog/petabytes-on-a-budget-v2-0revealing-more-secrets/ https://www.backblaze.com/blog/petabytes-on-a-budget-v2-0rev... We upgraded the Linux 64-bit OS from Debian 4 to Debian 5, but we no longer use JFS as the file system. We selected JFS years ago for its ability to accommodate large volumes and low CPU usage, and it worked well. However, ext4 has since matured in both reliability and performance, and we realized that with a little additional effort we could get all the benefits and live within the unfortunate 16 terabyte volume limitation of ext4. One of the required changes to work around ext4’s constraints was to add LVM (Logical Volume Manager) above the RAID 6 but below the file system. In our particular application (which features more writes than reads), ext4’s performance was a clear winner over ext3, JFS, and XFS.