7 ms·
ZFS on a single core RISC-V hardware with 512MB
- dark-star 5y agoBut can it do dedupe on such a box? I think the recommendation is still "1GB of RAM for each TB of storage" if you're using dedupe... I still have some boards with ~512mb RAM lying around (an UltraSPARC for example) that I'd love to re-purpose to a cheap NAS, just for the heck of doing it on a non-x86 platform....
- Wowfunhappy 5y agoYeah, I think the author may be mixing up recommendations for dedup vs non-dedup. The solution is always to not enable dedup, it's a niche feature that's not worthwhile outside of very specific scenarios.
- R0b0t1 5y agoThe speed you want them to run is a factor also. The rule of thumb hasn't applied for a while, he's right in noting that in the post.
- Wowfunhappy 5y agoI thought it does generally apply for dedup, though, because ZFS is then required to keep the dedup tables in memory?
- rdc12 5y agoIt isn't required (in the strict sense) to keep the dedup table in memory, the problem is that performance is dire when it doesn't. It would be pretty similar to virtual memory thrashing, when the table is not fully in memory.
- lazide 5y agoI’ve tried dedup out, and even with a large powerful box with a LOT of duplicate files (multi-TB repositories of media files which get duplicated several times due to coarse snapshotting from other less fancy systems), I get near zero deduplication. I think it was literally low single digits percents. ZFS dedup is block based, and actual block size varies depending on data feed rate for most workloads (zfs queues up async writes and merges them), so in practice once a file gets some non-zero block offset somewhere which happens all the time, even identical files don’t dedup.
- Wowfunhappy 5y agoWow, that’s worse than I realized! Honestly, this makes me wonder whether the feature should even exist in ZFS. Given the enormous hardware requirements and minimal savings... well, I’d be curious to hear if anyone has ever found a real use case.
- FullyFunctional 5y agoDoes anyone actually use dedup? I think even the OpenZFS documentation says compression is more useful in practice. If at all, dedup should be an offline feature, to be run as scheduled by the operator. My setup tries to get the absolute highest bandwidth and uses NVMe sticks in a stripe (I get my redundancy elsewhere), no compression, no dedup and yet can only hit ~ 3.5 GB/s reads (TrueNAS Core, EPYC 7443P, Samsung 980PRO, 256 GiB). I hope TrueNAS SCALE will perform better.
- bombcar 5y agoIt would be nice if ZFS was able to combine dedup and compression - basically be able to notice that a block/file/datastream was similar/identical to another one, and do compression along with a pointer ...
- willis936 5y agoZFS can have both features enabled at once. Though there is no clean way to disable either. Compression can be removed from files by rewriting them, but removing deduplication requires copying over all data to a fresh pool.
- michaelmrose 5y agoThere has not ever been a reason for memory to be correlated with storage capacity nor any reason to believe that such a correlation ought to exist. Nobody ever said well I plugged an 20TB external hard drive so I better plug in a few more sticks of RAM so that works. Dedup needs RAM in proportion to storage because for each duplicate block it maintains an entry in an in memory table.
- lazide 5y agoYou uh just kind of contradicted yourself? All file systems have metadata which is good to keep in memory. Building several 50+TB NAS boxes recently, it isn’t just ZFS either. And it isn’t some sort of linear performance penalties sometimes when you don’t have enough RAM. It can be kernel panics, exponential decay in performance, etc.
- michaelmrose 5y agoI didn't contradict myself at all virtually nobody needs dedup its not remotely worth the RAM cost for 99.9% of users. Can you quantify what you are saying. What OS/filesystem? What minimum RAM requirements for what amount of storage?
- lazide 5y agoThat’s a good point - I’ve seen free memory drop every time I’ve built the larger file systems (and not from just the cache), but I never tried to quantify it. And I don’t see any good stats or notes on it. seems like no one is building these larger systems on boxes small enough for it to matter, or at least google isn’t finding it.
- michaelmrose 5y agoAnother way of saying this is that RAM usage doesn't meaningfully scale with storage size in scope of storage systems encountered by actual non theoretical people because the minimum ram available on any system one encounters is sufficient to service the amount of storage that it is possible to use on said system.
- 2Gkashmiri 5y agohope this coupled with more tech on risc-v hardware can bring it to the level of raspbery pi with all the community and hardware devices and the accessories and all that. will it take a decade ? less?
- FullyFunctional 5y agoI hope it won't be a decade, but remember the (original) Raspberry Pi launched on a very mature part, with a _very_ mature (ancient) ISA. Outside the discount pricing, Intel has promised to tape out SiFive's P650. Revos, Tenstorrent, and others are also working on fast cores, but it'll be at least 2-3 years before they hit the market if at all. So far SiFive's dual issue in-order core (~ 40 GeekBench 5.4.1) (like on now-cancelled BeagleV) is the fastest chip you can buy as a lay person. The D1 (~ 32 GB 5.4.1) is cheaper but less powerful.
- mhh__ 5y agoIt will only happen if a company decides to try and be the next raspberry pi chip (or as a by-product like the OG broadcom chip from the Pi1). The actual chip designs, I think, are already there in terms of getting a high-performance risc-v chip built, but currently the market and tech stack is still getting they so we are still at the scaled-down-test phase in the high end market. FWIW though I don't see much reason to care about the ISA of the CPU beyond it being RISC, chances are it'll still be full of all kinds of closed source crap like the Pi.
- snvzz 5y ago>chances are it'll still be full of all kinds of closed source crap like the Pi. RISC-V's application profiles (like RVA22) do have requirements regarding some hardware that must be present (serial port with a specific interface), boot process and firmware interfaces. These are there to prevent an ARM-like messy situation.
- FullyFunctional 5y agoHa, this is awesome, thanks for checking that out. One point of note though, I'm pretty sure it would have been faster to build the kernel + OpenZFS on the Debian/RISC-V in QEMU. QEMU on decent hardware runs very fast and much faster than the D1. ADD: Geekbench 5.4.1 on RISC-V - under QEMU/Ryzen 9 3900XT: 82 - under QEMU/M1: 76 - Native D1: 32 (https://browser.geekbench.com/v5/cpu/13259016 https://browser.geekbench.com/v5/cpu/13259016) The M1 result is skewed because for some reason AES emulation is much faster on Ryzen. The rest of the integer stuff is faster on the M1, up to 30% faster.
- bombcar 5y agoAnyone have a low-power RISC or ARM hardware that supports many SATA ports?
- vorpalhex 5y agoWould also love to see this, even if it's experimental or beta. PiBox is the only contender I am aware of.
- mustache_kimono 5y agoDon't know why anyone hasn't made such a board a priority. Seems like a sweet spot.
- justaj 5y agoYeah I would love not being dependent on x86 and its horrid UEFI mess. Then again... I believe technically ARM (and even RISC-V) board manufacturers can also make something similar. Let's hope that doesn't become a thing though.
- yjftsjthsd-h 5y agoWhy not? I've been working from the view that UEFI is the one of the few things worth keeping from x86_64 - I do not want to deal with every board having its own special boot process that prevents one disk being able to boot different machines. And UEFI is already here and works, so why reinvent the wheel?
- snvzz 5y agoRISC-V took steps to prevent chaos. The application profiles (like RVA22) have a bunch of requirements on hardware that must be present, the boot process and interface the firmware offers to the OS (i.e. opensbi).
- NavinF 5y agoOk but does any of that work in practice? Can I write a random distro ISO to an SD card and expect it to work on any RISC-V board that has the same pointer size? Cause that’s how it works in PC land thanks to ACPI tables. I just checked the Ubuntu RISC-V download page and it has 2 different ISOs for 2 boards from the same vendor! And apparently those are the only 2 boards with ISOs available 0_o
- yjftsjthsd-h 5y ago> On Debian unstable (at time of writing, March 2022: bookworm/sid), OpenZFS is not yet available as debian package (zfsutils-linux). Hence you have to compile and build OpenZFS yourself. > Unfortunately OpenZFS seems not to support (yet) cross-compiling for the RISC-V platform, hence you have to build the kernel on the RISC-V board. Both of these seem like low hanging fruit, yes? If the upstream code supports it I'm surprised Debian doesn't already have packages, and cross-compiling isn't that special.
- ksec 5y agoCan we have Consumer NAS that support ZFS natively now? Synology? Even low end Synology Drive have 1GB DDR Memory and a Quad Core Cortex A55 CPU.
- aborsy 5y agoTrueNAS. If you don’t care about a GUI, you can easily set up any server.
- nix23 5y ago>Consumer NAS I other words, cheap unreliable plastic-boxes. You are much better of with TrueNAS and a HP MicroServer Gen10 Plus (with xeon and ecc)
- philjohn 5y agoThe top of the range QNAP rackmount models have ZFS natively.
- azubinski 5y agoYes, you can use a hair dryer to make roast chicken and you can run Linux kernel with ZFS support on chinese risc-v implementation. The world has become better, life has become more interesting.
- magicalhippo 5y agoContext: https://news.ycombinator.com/item?id=30669030 https://news.ycombinator.com/item?id=30669030