5 ms·

My biggest interest in DragonFlyBSD is hammer storage, while I of course appreciate the general hard work hw support and general usage does not suite most of my

by xte 8y ago

My biggest interest in DragonFlyBSD is hammer storage, while I of course appreciate the general hard work hw support and general usage does not suite most of my needs, especially when I start using NixOS/GuixSD with the wonderful idea of having declarative/functional systems that's the future of ANY OS IMVHO but hammer by itself is fantastic. It deserve a future not different than SSH. On GNU/Linux I "mimic" it with a classic poor man's solution (mdraid+LUKS+LVM+nilfs2), in the past I've used zfs first on OpenSolaris (SXDE/CE/Indiana) and after even on GNU/Linux but while remain a fantastic pioneer hammer being a logged fs is far superior. Thanks so for the hard work!

equalunique 8y ago
A applaud your repertoire of OS experience. Perhaps because HAMMER (+HAMMER2) is BSD licenced, it might have better potential as a Linux kernel module. I also recommend trying out Bcachefs. It's developed on Debian so that's your best bet to give it a shot.
- xte 8y ago
  I tried out bcachefs in the recent past but... Well... for now it's only a classic, usable, stable fs... All the nice features are only promises in development, no snapshot, no replication, nothing is there only a stable fs that seems no different than ext... On license of course there is a problem but if we have zfs ported well... I think it can be circumvented, only I fear that in GNU/Linux land (especially -Linux part) there are too many people that do not consider at all modern needs, most of them are programmer's with no sysadmining background, they do not even understand why we desperately need new storage and "Big&Powerful" are absolutely happy because actual paleolithic storage solution means business opportunities... Remember Andrew Morton with it infamous "rampant layer violation" against zfs or NetApp that ask SUN "to unfree" zfs to avoid disrupting storage market... IMO we have lost after Ubuntu ditch desktop the opportunity of a generic end-users GNU/Linux desktop and due to hw&sw evolution commercial-side we desperately need to innovate again as a FOSS community, DragonflyBSD is a very little substantial single-man-show project but it innovate, GNU keep innovate a bit despite a super-slow evolution and GuixSD is IMO the future of any OS, like it's "father" NixOS, the first IaC-builtin OSes in the world, but we need more, and with more manpower instead of wasting energy in evanescent crappy web-derived tech from GnomeShell extensions to Electron-based (cr)apps. Storage in that sense is the basis since on storage we can design package managers and init systems, so in turn installers and in turn the rest of the OS. Having a modern, powerful storage, something like Plan9 propose years ago, something like Hammer, zfs, nilfs2, stratis etc are a thing that need to evolve and spread in GNU/Linux world or we can't really evolve...
  - loeg 8y ago
    > Well... for now it's only a classic, usable, stable fs ... nothing is there only a stable fs that seems no different than ext... It's a CoW filesystem, which should provide some inherent crash/power-fail safety that ext does not have (at least, as usually configured — and data logging has huge overhead).
    - xte 8y ago
      Sorry my not-so-good English sometimes make me express concept in a convoluted/wrong way, on my previous post s/no/not much/ (different) than ext, it's an interesting project however today is only that IMO a thing to follow, prize, support but not use regularly apart if you want to help it's developments... On logging overhead... Well, yes and no, the logfs I tried do have overhead at garbage collection/cleaning but that overhead does not seems really important in practical terms. It demand a bit of careful design of storage layout and load prediction but nothing more than that. I do not see practical advantage for most server usage however I see significant advantage for some server usage and certainly for desktop usage. Also potential future development is certainly interested... Imaging a package manager integrated with a logfs how effective can be only in terms of "immutable servers/IaC" applications, imaging how NixOS/GuixSD generations can easily and instantly switch, how can you deep and easily analyze datasets and systems... Also overhead is there, a bit, today, but tomorrow can be significantly reduced if we have physical storage developed with logging in mind.
- loeg 8y ago
  Sure, HAMMER(2) is licensed for portability elsewhere. The trick would be adapting Dragonfly's VFS interface (and locking!) to your target platform. It has diverged from FreeBSD in ways that make that non-trivial, and Linux's system is even more different.
hawski 8y ago
Could you expand upon poor man's solution (mdraid+LUKS+LVM+nilfs2)? I'm mainly interested in NILFS2 part. For the long time I intend to use it. I thought about /home for me and/or my father and lately just for an archive partition (a bit like Fossil from Plan 9). I heard that there is a disagreement on whether it needs fsck or not. It does not have it and theoretically does not need it. But I heard about unrecoverable errors that probably could be fixed by hypothetical fsck. I can't find it now, it was some kind of a blog post that summed up the disagreement.
- xte 8y ago
  Well, for now is about an year with few machines (personal desktop, home server, laptop and few friends machines) on NixOS, root included [1] issueless. Various loads, even if not really "high" in server-like terms, very few not-clean shutdowns, mostly for testing, few fs resize (both in grow and shrink senses) + LVM resize [2], regular sync of different desktops via unison + converting in ss (snapshot) the latest cp (checkpoint) and back again after sync. The only real problem I found is cleaner process that's extremely slow so on frequently changing fs like root, home, var etc you need to tweak config a bit [3] to make cleaner far more aggressive or require really big free space to avoid fs full from time to time due to extra cp saved... That's really the sole problem I found. Respect of my ancient zfs-root setup is far lighter and logging is fantastic as a protection against accidental overwrite. Of course it's not as powerful as hammer and it UI is even more raw, for instance to diff a tree or a file you have to manually: - change relevant cp to an ss - mount the ss to an arbitrary location - diff live vol/snap against the other mounted A thing that limit usefulness a bit but considering that in terms of performance and overhead can be compared to a non-buggy btrfs with the overhead of xfs/jfs... It's really worth it! [1] you only need to use a custom iso, something super-easy on NixOS, mine is here http://ix.io/1wIQ http://ix.io/1wIQ the only really important part is boot.supportedFilesystems = [ "nilfs2" ]; boot.initrd.kernelModules = [ "crc32" "nilfs2" ]; you may choose zfs, f2fs, bcachefs, ... [2] simply for grow resize first the LVM lv and after nilfs2, without dimension it grow the maximum free space available; for shrink shrink nilfs2 a bit more than the target size, resize the LVM lv to the new desired size, grow nilfs2 without dimension to occupy the entire LV free space. All operation are live, with mounted fs, it can't be resized otherwise! [3] here http://ix.io/1wIS http://ix.io/1wIS a really aggressive config, that on SSD essentially does not crate any noticeable overhead, on plate disk it make them spin regularly but it's just a matter of noise for desktop usage.

by xte 8y ago

equalunique 8y ago

A applaud your repertoire of OS experience. Perhaps because HAMMER (+HAMMER2) is BSD licenced, it might have better potential as a Linux kernel module. I also recommend trying out Bcachefs. It's developed on Debian so that's your best bet to give it a shot.

xte 8y ago
I tried out bcachefs in the recent past but... Well... for now it's only a classic, usable, stable fs... All the nice features are only promises in development, no snapshot, no replication, nothing is there only a stable fs that seems no different than ext... On license of course there is a problem but if we have zfs ported well... I think it can be circumvented, only I fear that in GNU/Linux land (especially -Linux part) there are too many people that do not consider at all modern needs, most of them are programmer's with no sysadmining background, they do not even understand why we desperately need new storage and "Big&Powerful" are absolutely happy because actual paleolithic storage solution means business opportunities... Remember Andrew Morton with it infamous "rampant layer violation" against zfs or NetApp that ask SUN "to unfree" zfs to avoid disrupting storage market... IMO we have lost after Ubuntu ditch desktop the opportunity of a generic end-users GNU/Linux desktop and due to hw&sw evolution commercial-side we desperately need to innovate again as a FOSS community, DragonflyBSD is a very little substantial single-man-show project but it innovate, GNU keep innovate a bit despite a super-slow evolution and GuixSD is IMO the future of any OS, like it's "father" NixOS, the first IaC-builtin OSes in the world, but we need more, and with more manpower instead of wasting energy in evanescent crappy web-derived tech from GnomeShell extensions to Electron-based (cr)apps. Storage in that sense is the basis since on storage we can design package managers and init systems, so in turn installers and in turn the rest of the OS. Having a modern, powerful storage, something like Plan9 propose years ago, something like Hammer, zfs, nilfs2, stratis etc are a thing that need to evolve and spread in GNU/Linux world or we can't really evolve...
- loeg 8y ago
  > Well... for now it's only a classic, usable, stable fs ... nothing is there only a stable fs that seems no different than ext... It's a CoW filesystem, which should provide some inherent crash/power-fail safety that ext does not have (at least, as usually configured — and data logging has huge overhead).
  - xte 8y ago
    Sorry my not-so-good English sometimes make me express concept in a convoluted/wrong way, on my previous post s/no/not much/ (different) than ext, it's an interesting project however today is only that IMO a thing to follow, prize, support but not use regularly apart if you want to help it's developments... On logging overhead... Well, yes and no, the logfs I tried do have overhead at garbage collection/cleaning but that overhead does not seems really important in practical terms. It demand a bit of careful design of storage layout and load prediction but nothing more than that. I do not see practical advantage for most server usage however I see significant advantage for some server usage and certainly for desktop usage. Also potential future development is certainly interested... Imaging a package manager integrated with a logfs how effective can be only in terms of "immutable servers/IaC" applications, imaging how NixOS/GuixSD generations can easily and instantly switch, how can you deep and easily analyze datasets and systems... Also overhead is there, a bit, today, but tomorrow can be significantly reduced if we have physical storage developed with logging in mind.
loeg 8y ago
Sure, HAMMER(2) is licensed for portability elsewhere. The trick would be adapting Dragonfly's VFS interface (and locking!) to your target platform. It has diverged from FreeBSD in ways that make that non-trivial, and Linux's system is even more different.

hawski 8y ago

Could you expand upon poor man's solution (mdraid+LUKS+LVM+nilfs2)? I'm mainly interested in NILFS2 part. For the long time I intend to use it. I thought about /home for me and/or my father and lately just for an archive partition (a bit like Fossil from Plan 9). I heard that there is a disagreement on whether it needs fsck or not. It does not have it and theoretically does not need it. But I heard about unrecoverable errors that probably could be fixed by hypothetical fsck. I can't find it now, it was some kind of a blog post that summed up the disagreement.

xte 8y ago
Well, for now is about an year with few machines (personal desktop, home server, laptop and few friends machines) on NixOS, root included [1] issueless. Various loads, even if not really "high" in server-like terms, very few not-clean shutdowns, mostly for testing, few fs resize (both in grow and shrink senses) + LVM resize [2], regular sync of different desktops via unison + converting in ss (snapshot) the latest cp (checkpoint) and back again after sync. The only real problem I found is cleaner process that's extremely slow so on frequently changing fs like root, home, var etc you need to tweak config a bit [3] to make cleaner far more aggressive or require really big free space to avoid fs full from time to time due to extra cp saved... That's really the sole problem I found. Respect of my ancient zfs-root setup is far lighter and logging is fantastic as a protection against accidental overwrite. Of course it's not as powerful as hammer and it UI is even more raw, for instance to diff a tree or a file you have to manually: - change relevant cp to an ss - mount the ss to an arbitrary location - diff live vol/snap against the other mounted A thing that limit usefulness a bit but considering that in terms of performance and overhead can be compared to a non-buggy btrfs with the overhead of xfs/jfs... It's really worth it! [1] you only need to use a custom iso, something super-easy on NixOS, mine is here http://ix.io/1wIQ http://ix.io/1wIQ the only really important part is boot.supportedFilesystems = [ "nilfs2" ]; boot.initrd.kernelModules = [ "crc32" "nilfs2" ]; you may choose zfs, f2fs, bcachefs, ... [2] simply for grow resize first the LVM lv and after nilfs2, without dimension it grow the maximum free space available; for shrink shrink nilfs2 a bit more than the target size, resize the LVM lv to the new desired size, grow nilfs2 without dimension to occupy the entire LV free space. All operation are live, with mounted fs, it can't be resized otherwise! [3] here http://ix.io/1wIS http://ix.io/1wIS a really aggressive config, that on SSD essentially does not crate any noticeable overhead, on plate disk it make them spin regularly but it's just a matter of noise for desktop usage.