7 ms·
How to save the world with ZFS and 12 USB sticks: 4th anniversary video (2011)
- oofbey 1y agoIt’s funny which things have changed and which haven’t. That server which was super impressive for 2007 had 4 cores and 16GB of ram. Which is a reasonable small laptop for today. The 24TB of disk would still be pretty big though.
- bombcar 1y ago24 TB is available on a single drive now, though.
- temp0826 1y agoI haven't kept up with spinning rust drives so I had to take a look. Seagate has a couple 30 tb models now, crazy. Lot of eggs in one basket...and through the same ol 6gbit sata interface these must be a nightmare to format or otherwise deal with. Impressive nonetheless
- deleted 1y ago[deleted]
- wongarsu 1y agoIf you have one drive it feels like that, but if you throw 6+2 drives into a RAID6/raidz2. Sure, a full format can take 3 days (at 100 Megabytes/second sustained speed), but it's not like you are watching it. The real pain is fining viable backup options that don't cost an arm and a leg
- nubinetwork 1y agoIf your drives are only managing 100MB/s then something is wrong, SATA 3 should be at least 500MB/s.
- wongarsu 1y agoSATA 3 can move 500MB/s, but high-capacity drives typically can't. They are all below 300MB/s sustained even when shiny new. Look for example at the performance numbers quoted in these data sheets [1][2][3][4], all between 248 MiB/s and 272 MiB/s. Now that's still a lot faster than 100MB/s. But I have a lot of recertified drives, and while some of them make the advertised numbers some of them have settled at 100MB/s. You could argue that is something wrong with them, but they are in a raid and I don't need them to be fast. That's what the SSD cache is for. 1: Page 3 https://www.seagate.com/content/dam/seagate/en/content-fragments/products/datasheets/exos-x24/exos-x24-DS2080-2307US-en_US.pdf https://www.seagate.com/content/dam/seagate/en/content-fragm... 2: Page 2 https://www.seagate.com/content/dam/seagate/en/content-fragments/products/datasheets/exos-m-v2/exos-m-v2-DS2045-3-2506-en_US.pdf https://www.seagate.com/content/dam/seagate/en/content-fragm... 3: Page 2 https://www.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/product/data-center-drives/ultrastar-dc-hc600-series/data-sheet-ultrastar-dc-hc690.pdf https://www.westerndigital.com/content/dam/doc-library/en_us... 4: Page 7 https://www.seagate.com/content/dam/seagate/assets/products/nas-drives/ironwolf-pro-hard-drive/files/Seagate_IronWolf_Pro_SATA_Product_Manual_32-30-28-24TB_207967700B.pdf https://www.seagate.com/content/dam/seagate/assets/products/...
- nubinetwork 1y agoI had a 12 disk striped raidz2 array comprised of wd gold drives that could push 10Gbit/s over the network, while scrubbing, while running 10 virtual machines, and still had plenty of IO to play with. /shrug
- guenthert 1y agoSpinning rust drives tend to be much faster on the outer than the inner tracks.
- elygre 1y agoUnfortunately, I’m absolutely watching it :-( It happens whenever there is a progress indicator. I get obsessed with monitoring and verifying.
- Polizeiposaune 1y agoUnlike typical raid-5/6 parity, zfs / raidz doesn't require a format/parity initialization that writes all blocks of the disk before the pool can be used. You just need to write labels (at start & end of each disk) which is an attempt to confirm that the disk is actually as big as claimed.
- godelski 1y agoBtw, you can get refurbished ones for relatively cheap too. ~$350[0]. I wouldn't put that in an enterprise backup server, but pretty good deal for home storage if you're implementing raid and backups. [0] https://www.ebay.com/itm/306235160058 https://www.ebay.com/itm/306235160058
- bombcar 1y agoPrices have soared recently because AI eats storage as well as GPU; but tracking the data hoarder sites can be worthwhile. Seagate sometimes has decent prices on new.
- genpfault 1y ago> Seagate sometimes has decent prices on new. Make sure to check the "annual powered-on hours" entry in the spec sheet though, sometimes it can be significantly less than ~8766 hours.
- godelski 1y agoProbably a good time to mention systemd automount. This will auto mount and unmount drives as needed. You save on your energy bill but the trade off is that first read takes longer as drives need to mount. You need 2 files, the mount file and the automount file. Keep this or something similar as a skeleton file somewhere and copy over as needed # /etc/systemd/system/full-path-drive-name.mount [Unit] Description=Some description of drive to mount Documentation=man:systemd-mount(5) man:systemd.mount(5) [Mount] # Find with `lsblk -f` What=/dev/disk/by-uuid/1abc234d-5efg-hi6j-k7lm-no8p9qrs0ruv # See file naming scheme Where=/full/path/drive/name # https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/6/html/storage_administration_guide/sect-using_the_mount_command-mounting-options#sect-Using_the_mount_Command-Mounting-Options Options=defaults,noatime # Fails if mounting takes longer than this (change as appropriate) TimeoutSec=1m [Install] # Defines when to load drive in bootup. See `man systemd.special` WantedBy=multi-user.target # /etc/systemd/system/full-path-drive-name.automount [Unit] Description=Automount system to complement systemd mount file Documentation=man:systemd.automount(5) Conflicts=umount.target Before=umount.target [Automount] Where=/full/path/drive/name # If not accessed for 15 minutes drive will spin down (change as appropriate) TimeoutIdleSec=15min [Install] WantedBy=local-fs.target
- CTDOCodebases 1y agoThe new drives have dual actuators to improve performance: https://www.seagate.com/au/en/innovation/multi-actuator-hard-drives/ https://www.seagate.com/au/en/innovation/multi-actuator-hard... https://www.youtube.com/watch?v=5eUyerocA_g https://www.youtube.com/watch?v=5eUyerocA_g
- guenthert 1y agoI'm pretty sure that there were such drives more than twenty years ago (not popular though). I have to ask, what's the point today? The average latency goes down at most linearly with the number of actuators. One would need thousands to match SSDs. For anything but pure streaming (archiving), spinning rust seems questionable. Edit: found it (or at least one) "MACH.2 is the world’s first multi-actuator hard drive technology, containing two independent actuators that transfer data concurrently." World's first my ass. Seagate should know better, since it was them who acquired Connor Peripherals some thirty years ago. Connor's "Chinook" drives had two independent arms. https://en.wikipedia.org/wiki/Conner_Peripherals#/media/File:Conner_Peripherals_%22Chinook%22_dual-actuator_drive.jpg https://en.wikipedia.org/wiki/Conner_Peripherals#/media/File...
- namibj 1y agoThose HDDs, if single-actuator, spend around 2~4 MB of streaming potential per seek. That means, if you access files of exactly that size you'd "only" half your iops. HDDs are quite fine for data chunks in the megabytes.
- amy_petrik 1y ago[flagged]
- CTDOCodebases 1y agoThere are 36TB hard drives available. There are 122TB SSD drives now, though.
- UltraSane 1y agoKioxia has a 245TB SSD.
- CTDOCodebases 1y agoIf consumer SSD's are anything to go by I could probably buy two of those and never have to think about storage for the rest of my life.
- hugmynutus 1y agoBuddy, I have 24Tb HDDs in my pool today. If anything the opposite has occurred. HDD scaling has largely flattened. Going from 1986 -> 2014, HDD size increased by 10x every 5.3 years [1]. If anything we should have 100Tb+ drives if scaling kept going. I say this not as a but there have been directly implications for ZFS. All this data stuck behind an interface who's speed is (realistically after a file system & kernel involved) hard limited to 200MiB/s-300MiB/s. Recovery times sky rocket. As you simply cannot re-build parity/copy data. The whole reason stuff like draid [2] were created is so larger pools can recover in less than a day by doing sequential parity & hot-spairs loaded 1/N of each drives data ahead of time. --- 1. Not the most reliable source, but it is a friday afternoon https://old.reddit.com/r/DataHoarder/comments/spoek4/hdd_capacity_by_year_by_power_of_10_some_thoughts/ https://old.reddit.com/r/DataHoarder/comments/spoek4/hdd_cap... 2. https://openzfs.github.io/openzfs-docs/Basic%20Concepts/dRAID%20Howto.html https://openzfs.github.io/openzfs-docs/Basic%20Concepts/dRAI... for concept, for motivations & implementation details see -> https://www.youtube.com/watch?v=xPU3rIHyCTs https://www.youtube.com/watch?v=xPU3rIHyCTs
- godelski 1y agoNot quite that level, but you can get 8TB nvmes. You'll pay $500 a pop though...[0]. Weirdly that's the cheapest NewEgg lists for anything above 8TB and even SSDs are more expensive. It's a gen4 PCIe M.2 but a SATA SSD is more? It's better going the next bracket down but still surprising to me that the cheapest 4TB SSD is just $20 cheaper than the cheapest NVMe[1] (a little more and you're getting recognizable names too!) It kinda sucks that things have flatlined a bit, but still cool that a lot of this has become way cheaper. I think the NVMes at these prices and sizes really makes caching a reasonable thing to do for consumer grade storage [0] https://www.newegg.com/western-digital-8tb-black/p/N82E16820250270 https://www.newegg.com/western-digital-8tb-black/p/N82E16820... [1] https://www.newegg.com/p/pl?N=100011693%20600551612&Order=1 https://www.newegg.com/p/pl?N=100011693%20600551612&Order=1
- nick__m 1y agoIf your budget allows it you can get 120TB .5 dwdp ssd like that drive http://www.atic.ca/index.php?page=details&psku=319207 http://www.atic.ca/index.php?page=details&psku=319207
- ghshephard 1y agoNowadays most cloud customers jam their racks full of absolutely vanilla (brand is mostly irrelevant) 128 vCores / 64 physical cores, 512 GByte servers for ~$18k - w/100 GBit NICs. That Sun 4500, maxed out (with 10 Gbit/Nics), sold for $70k. ($110K in 2025 dollars). What's (still) super impressive was the 48 drives. Looking around -the common "Storage" nodes in rack these days seem to be 24x24TB CMR HDD + 2 7.68 TB NVME SDD (and a 960 GB Boot Disk) - I don't know if anyone really uses 48 drive systems commonly (outside the edge cases like Backblaze and friends)
- LaurensBER 1y agoI just grabbed a deal on a 8GB / 2 vCore VPS for 15 euro per year. It's absolutely insane how cheap hardware has become. At my first job we paid 6 figures for a 256GB ram machine. Now I see homelabers grabbing old servers with that much memory for their hobby projects.
- klinch 1y agoMay I ask where you got such a cheap VPS?
- jodrellblank 1y agoI usually look at lowendbox.com and Google “racknerd Black Friday vps” the cheapest are in the 10-20/year region, but don’t have that much RAM. It sounds a very good deal - unless they mean 8GB SSD storage, then it’s a normal sort of deal.
- LaurensBER 1y agohttps://lowendtalk.com/discussion/210253/deluxhost-net-j-series-vps-launch-unbeatable-value-from-7-yr-pre-order-ams/p1 https://lowendtalk.com/discussion/210253/deluxhost-net-j-ser... This offer is already gone but for sure there'll be a new one, VPS prices tend to trend downwards. I'm sure I'm comprising on reliability but there are going to be part of my hobby K8s cluster so a bit of downtime will make for a nice stress test ;)
- davekeck 1y agoSurprised that folks were still using RealPlayer in 2011
- zenmac 1y agoCan someone here on hn with more in deepth knowelege about ZFS commenting on why it is superior to EXT4 for example for file storage? Does each dir handle more children for example? Last time I read here HN ZFS still seem have edge case bugs. Has it matured now? Why don't distro such as debian etc just ship ZFS as the default instead of ext4?
- ksk23 1y agoI can only imagine the ZFS license is not free enough for Debian (not a rant).
- em-bee 1y agothe problem is less how free it is. as far as i know it is free enough. but the license is simply incompatible with the GPLv2 and therefore a combined program may not be distributed, because by distributing it you would violate the GPL license of the kernel.
- davisr 1y agoQuit being a lazy bum and use a search engine to answer your questions.
- zenmac 1y agoI don't know, I find HN is one place were we can actually engage real humans conversation on these type of tech related questions. Human dialog is often something that can results in more interesting discovery. Don't think you can get that with agenatic RL question answers feed back loop between models.
- rcxdude 1y agoIt's not a particularly interesting set of questions, though. If you looked up the features of zfs you would find what it has the ext4 doesn't. There's hundreds of articles, written by humans, on it. There's nothing that someone is going to helpfully put in an HN reply to these questions that you couldn't find faster by literally copy and pasting them into google, ignoring the AI summary.