9 ms·
The Birth of ZFS [video]
- rsync 11y agorsync.net supports ZFS send and receive to their cloud storage platform: http://www.rsync.net/products/zfsintro.html http://www.rsync.net/products/zfsintro.html
- bcantrill 11y agoHow have you dealt with the issue of byzantine streams? This is something that we have wanted to solve for a while, but it's a really nasty issue -- and at the recent OpenZFS Developers Summit we concluded that it wasn't practical. Instead, we're looking to tack more towards signed ZFS streams[1] -- though that work is only in the design stage. How has rsync.net solved this? [1] https://github.com/joyent/rfd/blob/master/rfd/0014/README.md https://github.com/joyent/rfd/blob/master/rfd/0014/README.md
- rsync 11y agoI think I know what you're referring to, but we've never called them "byzantine streams". Are you talking about snapshots that have been broken by extended attributes bugs, or "disappearing" snapshots ... and then what happens when you send/recv them ? Actually, either way, would you email info@rsync.net so we can chat over email ?
- bcantrill 11y agoI'm talking about a ZFS stream that has been deliberately corrupted and then sent to you to be received as a filesystem -- when it will in fact cause ZFS to panic (or worse). So I mean "byzantine" in the "characterized by deviousness" sense: someone actively trying to do harm to you. Or perhaps I have misunderstood the functionality that you offer?
- rsync 11y agoOk, that is not what I was thinking of. We have made no specific provisions for this. However, we were scared enough about "unknown unknowns" that we set up this aspect of our service such that every ZFS send/recv customer gets their own zpool inside their own bhyve. So it might be a DOS attack, but I don't think there is a data attack against rsync.net here. Comments ? Unrelated: I'm curious - when I speak of snapshots corrupted by extended attributes or "disappearing snapshots", do you know what I mean ? Maybe we should talk :)
- bcantrill 11y agoI don't know enough about bhyve or your implementation to be able to comment, but if you assume that any system that has ever received a ZFS stream from a user is entirely compromised, then perhaps you can work around it that way. I'm not aware of a disappearing snapshot problem; has there been something sent to the ZFS list about this? Would have been a good topic of discussion for the OpenZFS Developers Summit a few weeks ago... ;)
- rsync 11y agoYes, that's exactly the model - these users have their own zpool inside their own VM.
- shmerl 11y agoSo, are Oracle still developing their ZFS separately, or they switched to OpenZFS?
- jpgvm 11y agoSeparately. However their changes and OpenZFS changes are now effectively incompatible. Funnily enough is that OpenZFS is now the more advanced filesystem minus the clustering features Oracle integrated (shared disk clustering that is, not shared-nothing).
- acd 11y agoUsing zfs in production virtualization it´s fast and provides good integrity. You can basically get some of the hyped features of Docker the copyonwrite filesystem part with ordinary Linux virtualization.
- zf00002 11y agoI've been using it at home via FreeNAS after losing a bunch of old files to bitrot.
- StavrosK 11y agoHow would ZFS help there? If you don't scrub, you lose files whatever fs you use.
- abrookewood 11y agoHe never said he didn't scrub ... assuming he did (and he held 2+ copies), then the files are restored. Even if he only had a single copy, ZFS would at least tell you what files are corrupt, allowing you to restore them from backup.
- StavrosK 11y agoI guess that's true, I was thinking about the case of the redundant array (otherwise the files wouldn't be "lost").
- dmm 11y agoBitrot still affects traditional RAID. > As much as it affects ZFS, that's my point. That's not true. RAID protects you against disk failure but bitrot is caused by unrecoverable read failures. Check out this ars article: http://arstechnica.com/information-technology/2014/01/bitrot-and-atomic-cows-inside-next-gen-filesystems/ http://arstechnica.com/information-technology/2014/01/bitrot...
- giis 11y agoIt would be interesting to see comparison between ZFS and Btrfs, ReiserFS in terms of features and development effort. Any idea?
- mrweasel 11y agoWhy ReiserFS? It's pretty much a defunct filesystem at this point. It's not really in the same class as ZFS and Btrfs, if you want a third option maybe look at HammerFS? Although I think pretty much everyone is waiting for Hammer2. There was a talk "Why OpenBSD sucks" by Henning Brauer, where he complains that ZFS is a "kitchen sink" approach to a filesystem. He doesn't really explain further, but it would be interesting to hear someone elaborate that criticism.
- glass- 11y agoAlso see, Ted Unangst's post about ZFS where he lists some criticisms[0] which somewhat expands on Brauer's criticism, and he talks about when he was interviewed on BSD Now[1]. Keep in mind that their criticisms are mostly about it not being a good choice for OpenBSD, although some of them apply across the board. [0] http://www.tedunangst.com/flak/post/ZFS-on-OpenBSD http://www.tedunangst.com/flak/post/ZFS-on-OpenBSD [1] http://www.bsdnow.tv/episodes/2014_02_05-time_signatures http://www.bsdnow.tv/episodes/2014_02_05-time_signatures
- giis 11y agoThough ReiserFS is not actively developed, If I'm not wrong some Btrfs concepts ((like tail-packing?) derived from ReiserFS. Thanks for info about Henning Brauer talk, will check it out.
- jlouis 11y agoFar more interesting is the question: what error in the filesystem led to a situation where you lost data in production? It is very easy to crank out claims of features while not having really implemented said feature in a correct way. This leads to bugs which are hard to find and almost impossible to eradicate.
- nisa 11y ago
- davidguerrerom 11y agohola mas mas
- deleted 11y ago[deleted]
- geggam 11y agoCan we not post videos ? its really hard to skim the material and decide if i want to read more in depth.
- dijit 11y agoit's only 20 minutes and the content can be sped up.
- matt2000 11y agoI actually was coming here to say that it was nice to have links to interesting stuff on Youtube. I agree it can be jarring when you're trying to read stuff and get hit with a video - maybe a HN youtube channel would be cool or something.