8 ms·
It's kind of a shame that ZFS hasn't seen better adoption, and that btrfs has been pretty stagnant, and that in general next generation file systems are mostly
by tracker1 10y ago
It's kind of a shame that ZFS hasn't seen better adoption, and that btrfs has been pretty stagnant, and that in general next generation file systems are mostly still-born. It would be nice to see some improvements in this area as 10tb hdd's are eminent, and data management is important.
However, it's worth noting that RAID, especially in software on systems without ECC ram is less than ideal. Beyond this is the overhead of managing larger filesystems with ZFS. The DIY raid servers that support it have had some troublesome issues that I've experienced first hand.
It's likely a lot of these advantages have been displaced by the discontinuation of Apple's server projects as well as other fears. By similar note, I've always been somewhat surprised that NTFS hasn't been more popular for external media, as it's a pretty good fit there.
In the end, software at this level has been held back significantly by the patent hornets nest that exists in the world today. I truly hope that we can see some reform in this space, but with the likes of TPP and similar treaty negotiations around the world today, that is pretty unlikely. Eventually some countries will have to strike out, break these bad treaties and reign in IP law. The vast majority of software is undeserving of patent protection. Just as software really shouldn't have copyright law that lasts for decades. It's a shame all around.
- ryao 10y agoThere is nothing special about storage that makes the absence of ECC in a computer any more of a bad idea than it already is. Building machines without ECC is a bad idea.
- tracker1 10y agoWhen writing redundant (parity data) across multiple storage devices, you're increasing the chance of error without ECC. While there's still a chance/risk with single drive, the effects of such error are not as significant.
- empthought 10y agoThe effects are exactly as significant either way: corrupt data is written to disk. Whether the corrupt data is written one time or several times is immaterial.
- jonathankoren 10y agoMay I introduce you to error correction? Having an error on one disk, versus multiple disks in a RAID totally matters.
- empthought 10y agoHow do you figure? If the single disk has bad data written to it, it contains bad data. If the RAID has bad data written to it, it contains bad data. Either way, the bad data is bad, because it was undetectably corrupted in memory before being written out.
- jonathankoren 10y agohttps://www.youtube.com/watch?v=osS9EaKNSI4 https://www.youtube.com/watch?v=osS9EaKNSI4 Serious question: Do they not teach Hamming codes in undergrad anymore?
- empthought 10y agoDo you not understand the problem here? The problem isn't detecting data corruption in storage, which parity checking also helps with. The problem is detecting data corruption in memory. ECC only matters after irrecoverable data loss _in memory_ has occurred. Without ECC the system will continue to operate; with ECC the system will halt without writing data at all, or correct it before storage even comes into play. Writing corrupt data once or writing it a thousand times redundantly with parity data looks exactly the same, if it's allowed to happen. It's a valid write of invalid data. Either way, the data you have on your cheap laptop drive or your six-figure storage array is corrupt. This is why the ancestor commenter is correct about the storage technology being irrelevant to the choice to use or not use ECC. I haven't been in school for many years. They taught Hamming codes, but they also taught garbage-in, garbage-out.
- semi-extrinsic 10y ago> Building machines without ECC is a bad idea. I'd say it depends on the use case. For business storage servers, especially "mission-critical", sure. (Then again, this data was created somewhere; presumably mainly on laptops without ECC. Why is that not a bad idea?) But for a home storage server, ECC is no more critical than buying a UPS, shielded SATA cables, enterprise-grade drives, a high quality power supply, etc. etc. Together these upgrades will easily double the cost of the storage server. Since you'll want an off-site backup solution anyway in case of disaster, why bother paying for an expensive server that only reduces the chance of needing to restore from backup from 0.001% to 0.00001%?
- wyager 10y agoAny work we do to improve the reliability of consumer software is limited by the reliability of the underlying machine. If everyone used ECC RAM, it would be marginally more expensive than regular RAM, but it would let us (theoretically) cut the number of real-life crashes by a huge degree. I think if every computer system were like this, it would have non-obvious positive effects on society (like the ability to safely give computers increased responsibility).
- semi-extrinsic 10y ago> it would let us (theoretically) cut the number of real-life crashes by a huge degree Do you have a source for that claim? I know people have done tests with running large molecular simulations on hundreds of GPUs with/without ECC for days on end, and who've come to the conclusions that ECC probably isn't worth it [1]. And this makes sense. Most of a program's memory usage is data. Thus the rare event of a bitflip is most likely to occur in data. Such a bitflip doesn't cause a crash in a consumer system; maybe it changes the color of one pixel in one frame of a video file, or adds some other tiny noise somewhere. [1] http://dl.acm.org/citation.cfm?id=2484774 http://dl.acm.org/citation.cfm?id=2484774
- wyager 10y agohttp://research.google.com/pubs/pub35162.html http://research.google.com/pubs/pub35162.html
- marcosdumay 10y agoFully agree. But notice that a bit-error in a text file in plan storage has some consequences. That same bit-error in a text file in a RAID (or encrypted storage) has very different consequences. Some bad ideas are just worse than others.
- tw04 10y ago10TB? 15TB SSDs are already shipping with much larger not too far out: http://arstechnica.com/information-technology/2016/03/samsungs-monstrous-15tb-ssd-is-now-shipping/ http://arstechnica.com/information-technology/2016/03/samsun...
- tracker1 10y agoOkay, wasn't aware of that option... mainly going by 8TB WD reds on amazon, and figuring 12-12TB should be around soon.
- zippergz 10y agoI'd be curious to hear about the issues you've had with zfs. I'm running it on a fileserver in my house, and it's been solid for a few years, but my use isn't very demanding. I'd like to know what problems I should be keeping an eye out for (aside from the usual stuff like drive failures, etc.).
- tracker1 10y agoIt wasn't ZFS perse... the software I was using wouldn't properly enable the drives configured as hot spares... that and a few errors (I didn't use ECC), and a couple bad drives, and everything (11TB of data) lost. All the really important stuff was well backed up, as was my audio library. I never did re-rip all my DVDs & BluRays again since that happened.
- zippergz 10y agoGood info, thanks. I'll admit, I'm pretty terrified of having to replace one of the drives in my array. I've replaced dozens or maybe even hundreds of drives in Sun, SGI and DEC storage arrays over the years, but I have a lot less confidence in my homebrew setup. Fortunately, I have regular complete backups of everything there, so even if I lost the whole array, all I'd really lose is the time involved in fixing it, but that's still painful.
- tracker1 10y agoI felt the same way, which was why I went Raid-Z2 (dual parity) with two hot spares configured... even had a spare controller next to the box. Still, lost it all.
- brokenmachine 10y agoDamn, how did that happen? When the first drive failed, why didn't the hot spare get activated? What software was it that didn't work properly? This worries me because Raid-Z2 with two hot spares is exactly the system that I was planning to set up.
- arca_vorago 10y agoI haven't really gotten a chance lately to dig into it as much as I want, but I have heard that BTRFS is actually doing pretty well, with lots of updates in the last year that fix the middle-ground bugs, with a few big bugs still out there (such as raid6 mode in general). The main big data file system I am keeping my eye on is HAMMER2... but development is slow. I really like a lot of things Matthew Dillon is doing there though, especially the network stack. If I were doing a new isp or wisp I would probably be basing it on dflybsd.
- tracker1 10y agoI mainly mean that it'll be a while before btrfs stabilizes and even then, it's been how many next-gen file systems in floss land each with critical issues, usually non-technical in nature in even the past 5 years?
- ktRolster 10y agoKind of weird how Linux filesystems always seem to have problems....it seems like that's the sort of problem open source would do a reasonable good job at.
- TheCondor 10y agoDo they have more problems or more users in many more different situations? And then they are vocal... ZFS has this sterling reputation but a fair number of people have actually lost data to it. Some of it is bad hardware,orb ad choices in hardware, some of it is bad administration, it's still happened.
- ktRolster 10y agoLinus isn't very happy: http://www.linux-magazine.com/Online/News/Linus-Torvalds-Upset-over-Ext3-and-Ext4 http://www.linux-magazine.com/Online/News/Linus-Torvalds-Ups... I don't expect any filesystem to eliminate the need for actual backups, but
- honkhonkpants 10y agoZFS isn't catching on because it solves a 1990s-era problem that people just don't care that much about right now. People are focused on cloud for the moment. Cloud storage doesn't require the application developer or user to care about the filesystem, and clouds are not implemented with fancy filesystems on the individual hosts, but rather using fancy facility-wide filesystems that have better management and durability features than ZFS offers. ZFS would have been totally brilliant on a 90s-style multiuser enterprise or higher education deployment, though. It just missed its window.
- oofabz 10y agoDistributed filesystems are backed by local filesystems. The local filesystem can be ZFS with all of its benefits even if you also have a fancy facility-wide filesystem running on top of it.
- kalleboo 10y ago> I've always been somewhat surprised that NTFS hasn't been more popular for external media, as it's a pretty good fit there. Isn't that because NTFS is Microsoft-proprietary? AFAIK all the existing third-party implementations are reverse-engineered and hence not 100% safe. Meanwhile even HFS+ has a public specification published by Apple.
- tracker1 10y agoYeah, it is... that said, I do wish MS (NTFS) and Oracle (ZFS) were more permissive wrt patents and would do patent releases. Beyond that, it would be great if MS would even release open documentation on NTFS. I still think it's a pretty great option for portable storage.
- binarycrusader 10y agoThe CDDL license explicitly provides a patent grant and since the ZFS code was released under the CDDL, I'm not sure what you're asking for.
- technion 10y agoNTFS I ran into a bug just today that I'm still struggling to believe. A large Oracle server fell over, just reporting "IO errors". The issue was ultimately traced to NTFS's handling of fragmentation[0]. I've gone about ten years thinking running a defrag - particular on a virtual disk sitting on a SAN in a RAID array - made no sense. Today's downtime says otherwise. It's one thing to perform poorly. But seeing services just die with disk write errors on a disk that's 30% full has left me with a very low opinion of NTFS. Our Oracle person sounds like a desktop maintenance person from the 90's making an issue out of "the importance of weekly defrags". [0] https://support.microsoft.com/en-au/kb/967351 https://support.microsoft.com/en-au/kb/967351
- Spooky23 10y agoIn my DBA days in the 90s/2000s, this sort of issue was usually a lousy DBA that had extent sizes set too small. Windows can clearly handle big databases. The people getting paid to run the platform need to have a clue about the platforms they are running.
- coredog64 10y agoXFS is subject to fragmentation as well. Allegedly you can minimize it with some careful planning and mount options, but it seems easier just to enable the scheduled tsk that can be preempted.
- vetrom 10y agoIt took btrfs the same thing (maybe only over a year or two though) to realize they needed automatically triggered rebalance, which is essentially a very similar operation.
- rsync 10y ago"It's kind of a shame that ZFS hasn't seen better adoption" We[1] are full steam ahead! We transitioned all of rsync.net, globally, to ZFS starting in ate 2012. After that, we began offering zfs send and receive (over ssh, of course) into rsync.net filesystems[2]. It's working wonderfully. We have many, many customers that do extremely efficient backups of very large zpools/snapshots into the rsync.net location of their choice. It's been tested and reviewed favorably in the press.[3] [1] rsync.net [2] http://www.rsync.net/products/zfsintro.html http://www.rsync.net/products/zfsintro.html [3] https://0x.co/G36YPW https://0x.co/G36YPW
- rsync 10y agoYep, used an "Oh By Code" as a shortener for that arstechnica link ... URL shortening is a secondary use-case: https://news.ycombinator.com/item?id=11896717 https://news.ycombinator.com/item?id=11896717
- pklausler 10y agoeminent != imminent != immanent