4 ms·

DragonFlyBSD is slowly but steadily becoming more attractive as a Linux/FreeBSD alternative. 4.6 has good support for various newer Intel GPUs, hammer is stable

DragonFlyBSD is slowly but steadily becoming more attractive as a Linux/FreeBSD alternative. 4.6 has good support for various newer Intel GPUs, hammer is stable and the SMP and Networking performance is stellar. I had a 2.8TB backup archive that I have repeatedly tried to dedup with ZFS and the realtime while-you-are-writing architecture of ZFS dedup just kills the performance so badly that I have never actually succeeded going over the full dataset. Past week I made a 4.6-rc2 based DragonFly VM, attached a 3TB disk with RDM, formatted it with HAMMER, started rsync of the backed up data and ran hammer dedup every 30 min - I am now at 1.9TB used after the whole thing was done and a final dedup ran - no noticeable speed drops and I got around 900GB back!

rincebrain 10y ago
Did you try something like bup? (Of course, depending on your use case, that might not work out so well.) Not that I have anything but respect for DFBSD, but I'd have suggested trying a userland archiving tool with dedup before spinning up a VM + FS in it.
- blinkingled 10y ago
  Yeah I did try it on btrfs IIRC - but it's far too complicated (as opposed to hammer dedup where everything is included and it just works) and doesn't cater to my use case too well - I have multiple machines backing files up to a SMB share with lot of potential for duplicate data. I also need to be able to access those backed up files back over SMB. Just making a HAMMER mount point, running dedup on it via cron job at night and exporting it via Samba does everything I need.
  - rincebrain 10y ago
    Yeah, that's much nicer for your use case than bup. (I should point out that Windows apparently has similar after-the-fact dedup capabilities on NTFS in Server 2012/R2 and up [1], though I suspect you'll find DFBSD much easier to run with low overhead in a VM.) [1] - https://msdn.microsoft.com/en-us/library/hh769303%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396 https://msdn.microsoft.com/en-us/library/hh769303%28v=vs.85%...
    - blinkingled 10y ago
      It's not present in the R2 essentials sku though - so cost is also a major factor.

rincebrain 10y ago

Did you try something like bup? (Of course, depending on your use case, that might not work out so well.) Not that I have anything but respect for DFBSD, but I'd have suggested trying a userland archiving tool with dedup before spinning up a VM + FS in it.

blinkingled 10y ago
Yeah I did try it on btrfs IIRC - but it's far too complicated (as opposed to hammer dedup where everything is included and it just works) and doesn't cater to my use case too well - I have multiple machines backing files up to a SMB share with lot of potential for duplicate data. I also need to be able to access those backed up files back over SMB. Just making a HAMMER mount point, running dedup on it via cron job at night and exporting it via Samba does everything I need.
- rincebrain 10y ago
  Yeah, that's much nicer for your use case than bup. (I should point out that Windows apparently has similar after-the-fact dedup capabilities on NTFS in Server 2012/R2 and up [1], though I suspect you'll find DFBSD much easier to run with low overhead in a VM.) [1] - https://msdn.microsoft.com/en-us/library/hh769303%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396 https://msdn.microsoft.com/en-us/library/hh769303%28v=vs.85%...
  - blinkingled 10y ago
    It's not present in the R2 essentials sku though - so cost is also a major factor.