8 ms·

DragonflyBSD 6.0

176 points by vermaden 5y ago

2pEXgD0fZ5cF 5y ago
I am always fascinated by DragonflyBSD and catch myself peeking everytime some news about it pop up. However I have not had a chance or excuse to fully try it myself yet. A question on their repository and packaging: How do they handle it? Are they simply compatible with FreeBSD packages so most packages from there are just available on DragonflyBSD as well?
- jayp1418 5y ago
  DragonflyBSD uses pkgsrc's and Dsynth package management system as far as I know but mailing list of DragonflyBSD can answer better.
  - stock_toaster 5y ago
    I believe DFBSD uses pkgng, not pkgsrc.
    - nix23 5y ago
      True. >It is based on FreeBSD's Ports Collection. https://www.dragonflybsd.org/docs/howtos/HowToDPorts/ https://www.dragonflybsd.org/docs/howtos/HowToDPorts/
harikb 5y ago
In case anyone wants a background on what Dragonfly vs FreeBSD > The ultimate goal of the DragonFly project at its inception was to provide native clustering support in the kernel. ... While full cache coherency is no longer a top level goal, filesystem coherency is, and that direction continues to guide the project in a number of ways. > DragonFly BSD was forked from FreeBSD 4.8 in June of 2003, by Matthew Dillon. The project was originally billed as "the logical continuation of the FreeBSD 4.x series", as quoted in the announcement, but this description has long since become obsolete. From a performance perspective DragonFly's only real competitor these days is Linux Are there any real world success stories with this?
- webaholic 5y ago
  Even the performance claims do not really hold up anymore. Freebsd in general performs better and not surprising since it has so many more resources.
  - spijdar 5y ago
    Yeah, with all due respect to the developers (because getting as far as they have is a huge accomplishment!) the biggest feature of DFly I can identify is HAMMER, which is most interesting if you don't want to use ZFS for license reasons. I remember some of the old goals were things like process snapshotting and transferring, as in saving a process state and transferring it to another machine. It seems most of these ambitions ended up discarded, which is a shame, since it's increasingly becoming a "FreeBSD, but with fewer features, more incompatibilities, and marginally worse performance" :(
    - rektide 5y ago
      HAMMER's ability to snapshot an arbitrary directory is still something I deeply deeply deeply respect & wish btrfs or zfs had. I'm not sure the status but HAMMERFS also supposedly was going to have solid transparent encryption, which is something I wish btrfs did. I'd really like to be able to take a fs snapshot, send it to a friend, have them load the snapshot volume, but be unable to access it. But as soon as I did show up at their place with my keys, we could instantly unlock it & use it. I understand ZFS does a better job of this all, but balancing ZFS pools & adding drives has always sounded impossibly hard compared to btrfs's "it just works" adding-drives to RAID scenarios.
      - spijdar 5y ago
        Ahh, that does sound like a nifty feature! I didn't mean to so strongly imply HAMMER was a knock-off ZFS, more that it fills a similar niche, similar enough that ZFS's dominance kind of takes away from the "wow factor" of HAMMER. It makes more sense (to me anyway) when you consider that at the time HAMMER was first released, the Linux port of ZFS had only just begun, and FreeBSD had just released their first port of ZFS. Btrfs was also still an out-of-tree patch then, only mainlined in 2009. (not to mention ZFS as a whole probably seemed much more shaky around 2010 with Sun's death-by-Oracle and OpenSolaris getting butchered) I think back then, having a clean and functional "recreation" of ZFS with all the design bloat removed seemed really promising. Now, it's just a nice FS competing with other nice FSes, except with the downside of being on an "exotic OS". Shame :(
jarrell_mark 5y ago
A bit unrelated but did you all know that PlayStation's and Switch's OSs are both based on FreeBSD to varying extents?
- 9front 5y ago
  Lots of older Dell switches run NetBSD as NOS. Newer ones use Linux.
- mrweasel 5y ago
  That does make sense if you're a large corporation. You might not want to develop an OS from scratch, but Windows and WXWorks requires a license. You could go Linux, but what about the GPL? You might be okay, but why risk it. That leaves you with the choice of any of the BSDs.
  - niea_11 5y ago
    Sony already uses Linux in some of their TVs. https://oss.sony.net/Products/Linux/TV/KD-43XE7000_v8902.html https://oss.sony.net/Products/Linux/TV/KD-43XE7000_v8902.htm...
- mushufasa 5y ago
  yes. and I imagine those choices were more likely due to the permissive license than for any particular technical reason.
  - jayp1418 5y ago
    I don't think it was just license : https://klarasystems.com/articles/deep-diving-into-the-strengths-of-freebsd/ https://klarasystems.com/articles/deep-diving-into-the-stren... Otherwise they could have taken NetBSD, OpenBSD or other BSD
- Cu3PO42 5y ago
  I can't speak to the PlayStation, but this claim doesn't really hold for the Switch. The Switch's OS "Horizon" borrows BSD's networking stack, but then again: which OS doesn't? [0] Nintendo has written their own kernel, which is fully reverse engineered and re-implemented at this point. [1] [0] https://en.wikipedia.org/wiki/Nintendo_Switch_system_software#Open_source_components https://en.wikipedia.org/wiki/Nintendo_Switch_system_softwar... [1] https://github.com/Atmosphere-NX/Atmosphere/tree/master/libraries/libmesosphere https://github.com/Atmosphere-NX/Atmosphere/tree/master/libr...
tw04 5y ago
I'll preface this with: this isn't a derogatory question, I just don't know a better way to phrase it. Out of curiosity, who uses this? I assume given how long it's been around there must be a market segment or a geo local where it's popular (like SLES has large penetration in Europe and SAP). I know multiple closed source vendors that have built products on net/open/freeBSD. Dragonfly is the one distro I've never once run into in the wild.
- gjs278 5y ago
  I run garyshood.com on it
- nix23 5y ago
  I used it for some time ~3y as a backupserver and was happy with it, just test it out.
- enriquto 5y ago
  I have a Dragonfly install as part of my "pet" test farm. The build environment is similar to the other BSD, but from time to time it triggers strange portability bugs in my codes that I love to fix.
truth_seeker 5y ago
Excuse me what ? If this is really true, why isn't it more widely adopted than Ubuntu for server side deployment ? then following text is copied from "features" page of official site >EXTREME SCALING DragonFly will autotune kernel resources and scaling metrics such as kernel hash-tables based on available memory. The autoscaling has reached a point where essentially all kernel components will scale to extreme levels. Process and thread components now scale to at least a million user processes or threads, given sufficient physical memory to support that much (around 128GB minimum for one million processes). The PID is currently limited to 6 digits, so discrete user processes are capped at one million, but the (process x thread) matrix can conceivably go much higher. Process creation, basic operation, and destruction have been tested to 900,000 discrete user processes. File data caching scales indefinitely, based on available memory. A very generous kern.maxvnodes default allows the kernel to scale up to tracking millions of files for caching purposes. IPI signaling between CPUs has been heavily optimized and will scale nicely up to the maximum hardware thread limit (256 cpu threads, typically in a 128-core/256-thread configuration). Unnecessary IPIs are optimized out, and the signaling of idle cpus can be further optimized via sysctl parameters. All major kernel resource components are fully SMP-aware and use SMP-friendly algorithms. This means that regular UNIX operations that manipulate PIDs, GIDs, SSIDs, process operations, VM page faults, memory allocation and freeing, pmap updates, VM page sharing, the name cache, most common file operations, process sleep and wakeup, and locks, are all heavily optimized and scale to systems with many cpu cores. In many cases, concurrent functions operate with no locking conflicts or contention. The network subsystem was rewritten pretty much from the ground-up to fully incorporate packet hashes into the entire stack, allowing connections and network interfaces to operate across available CPUs concurrently with little to no contention. Pipes and Sockets have also been heavily optimized for SMP operation. Given a machine with sufficient capability, hundreds of thousands of concurrent TCP sockets can operate efficiently and packet routing capabilities are very high. The disk subsystem, particularly AHCI (SATA) and NVMe, are very SMP friendly. NVMe, in particular, will configure enough hardware queues such that it can dispatch requests and handle responses on multiple cpus simultaneously with no contention. The scheduler uses per-cpu algorithms and scales across any number of cpus. In addition, the scheduler is topology-aware and gains hints from whatever IPC (Inter-Process Communications) occurs to organize active processes within the cpu topology in a way that makes maximum use of cache locality. Load is also taken into account, and can shift how cache locality is handled. The kernel memory manager is somewhat NUMA aware. Most per-cpu operations use NUMA-local memory allocations. User memory requests are also NUMA aware, at least for short-lived user programs. Generally speaking, the scheduler will try to keep a process on the same cpu socket but ultimately we've determined that load balancing is sometimes more important. CPU caches generally do a very good job of maximizing IPC (Instructions Per Clock). Because memory management is fully SMP-aware, a multi-core system can literally allocate and free memory at a rate in the multiple gigabytes/sec range. Generally very high concurrency with very low kernel overhead. The kernel can handle just about any load thrown at it and still be completely responsive to other incidental tasks. Systems can run efficiently at well over 100% load. Supports up to 4 swap devices for paging and up to 55TB (Terabytes) of configured swapspace. Requires 1MB of physical ram per 1GB of configured swap. When multiple swap devices are present, I/O will be interleaved for maximum effectiveness. The paging system is extremely capable under virtually any load conditions, particularly when swap is assigned to NVMe storage. Concurrent page-in across available cpus, in particular, works extremely well. Asynchronous page-out. Extended filesystem data caching via the swapcache mechanism can operate as an extended (huge) disk cache if desired, and/or used to increase the apparent total system memory.
mikece 5y ago
Dragonfly does not support ARM64 (aarch64) but only x86-64???
- pikrzyszto 5y ago
  yes - one of the reasons behind Dragonfly conception was that it's hard to write high-performant code for all architectures at the same time. Ditching ARM64 and others means much cleaner codebase and less effort to maintain all of this.
  - webaholic 5y ago
    If portability of other OSes has taught us anything, it is that the interfaces are much cleaner if you are to support multiple architectures. I think it comes down to having people/companies interested in porting it to ARM64, not about cleaner codebases.
    - nix23 5y ago
      You are absolutely correct, however DragonFly focused since the beginning on x86, that was the target platform for a cluster and the architecture every dev had. Maybe it's time to rethink that.
ncmncm 5y ago
Still validating with md5, in 2021? DragonflyBSD distinguishes itself by supporting a variant of the Apollo Aegis, and later Domainix, practice of expanding environment variable names in symbolic link text. Of course it is more clumsy than in Aegis, for the sake of compatibility with other BSDs and systems that might, by long odds, happen to have what look like environment variable references in their symbolic links, but aren't.

176 points by vermaden 5y ago

I am always fascinated by DragonflyBSD and catch myself peeking everytime some news about it pop up. However I have not had a chance or excuse to fully try it myself yet. A question on their repository and packaging: How do they handle it? Are they simply compatible with FreeBSD packages so most packages from there are just available on DragonflyBSD as well?

jayp1418 5y ago
DragonflyBSD uses pkgsrc's and Dsynth package management system as far as I know but mailing list of DragonflyBSD can answer better.
- stock_toaster 5y ago
  I believe DFBSD uses pkgng, not pkgsrc.
  - nix23 5y ago
    True. >It is based on FreeBSD's Ports Collection. https://www.dragonflybsd.org/docs/howtos/HowToDPorts/ https://www.dragonflybsd.org/docs/howtos/HowToDPorts/

harikb 5y ago

In case anyone wants a background on what Dragonfly vs FreeBSD > The ultimate goal of the DragonFly project at its inception was to provide native clustering support in the kernel. ... While full cache coherency is no longer a top level goal, filesystem coherency is, and that direction continues to guide the project in a number of ways. > DragonFly BSD was forked from FreeBSD 4.8 in June of 2003, by Matthew Dillon. The project was originally billed as "the logical continuation of the FreeBSD 4.x series", as quoted in the announcement, but this description has long since become obsolete. From a performance perspective DragonFly's only real competitor these days is Linux Are there any real world success stories with this?

webaholic 5y ago
Even the performance claims do not really hold up anymore. Freebsd in general performs better and not surprising since it has so many more resources.
- spijdar 5y ago
  Yeah, with all due respect to the developers (because getting as far as they have is a huge accomplishment!) the biggest feature of DFly I can identify is HAMMER, which is most interesting if you don't want to use ZFS for license reasons. I remember some of the old goals were things like process snapshotting and transferring, as in saving a process state and transferring it to another machine. It seems most of these ambitions ended up discarded, which is a shame, since it's increasingly becoming a "FreeBSD, but with fewer features, more incompatibilities, and marginally worse performance" :(
  - rektide 5y ago
    HAMMER's ability to snapshot an arbitrary directory is still something I deeply deeply deeply respect & wish btrfs or zfs had. I'm not sure the status but HAMMERFS also supposedly was going to have solid transparent encryption, which is something I wish btrfs did. I'd really like to be able to take a fs snapshot, send it to a friend, have them load the snapshot volume, but be unable to access it. But as soon as I did show up at their place with my keys, we could instantly unlock it & use it. I understand ZFS does a better job of this all, but balancing ZFS pools & adding drives has always sounded impossibly hard compared to btrfs's "it just works" adding-drives to RAID scenarios.
    - spijdar 5y ago
      Ahh, that does sound like a nifty feature! I didn't mean to so strongly imply HAMMER was a knock-off ZFS, more that it fills a similar niche, similar enough that ZFS's dominance kind of takes away from the "wow factor" of HAMMER. It makes more sense (to me anyway) when you consider that at the time HAMMER was first released, the Linux port of ZFS had only just begun, and FreeBSD had just released their first port of ZFS. Btrfs was also still an out-of-tree patch then, only mainlined in 2009. (not to mention ZFS as a whole probably seemed much more shaky around 2010 with Sun's death-by-Oracle and OpenSolaris getting butchered) I think back then, having a clean and functional "recreation" of ZFS with all the design bloat removed seemed really promising. Now, it's just a nice FS competing with other nice FSes, except with the downside of being on an "exotic OS". Shame :(

jarrell_mark 5y ago

A bit unrelated but did you all know that PlayStation's and Switch's OSs are both based on FreeBSD to varying extents?

9front 5y ago
Lots of older Dell switches run NetBSD as NOS. Newer ones use Linux.
mrweasel 5y ago
That does make sense if you're a large corporation. You might not want to develop an OS from scratch, but Windows and WXWorks requires a license. You could go Linux, but what about the GPL? You might be okay, but why risk it. That leaves you with the choice of any of the BSDs.
- niea_11 5y ago
  Sony already uses Linux in some of their TVs. https://oss.sony.net/Products/Linux/TV/KD-43XE7000_v8902.html https://oss.sony.net/Products/Linux/TV/KD-43XE7000_v8902.htm...
mushufasa 5y ago
yes. and I imagine those choices were more likely due to the permissive license than for any particular technical reason.
- jayp1418 5y ago
  I don't think it was just license : https://klarasystems.com/articles/deep-diving-into-the-strengths-of-freebsd/ https://klarasystems.com/articles/deep-diving-into-the-stren... Otherwise they could have taken NetBSD, OpenBSD or other BSD
Cu3PO42 5y ago
I can't speak to the PlayStation, but this claim doesn't really hold for the Switch. The Switch's OS "Horizon" borrows BSD's networking stack, but then again: which OS doesn't? [0] Nintendo has written their own kernel, which is fully reverse engineered and re-implemented at this point. [1] [0] https://en.wikipedia.org/wiki/Nintendo_Switch_system_software#Open_source_components https://en.wikipedia.org/wiki/Nintendo_Switch_system_softwar... [1] https://github.com/Atmosphere-NX/Atmosphere/tree/master/libraries/libmesosphere https://github.com/Atmosphere-NX/Atmosphere/tree/master/libr...

tw04 5y ago

I'll preface this with: this isn't a derogatory question, I just don't know a better way to phrase it. Out of curiosity, who uses this? I assume given how long it's been around there must be a market segment or a geo local where it's popular (like SLES has large penetration in Europe and SAP). I know multiple closed source vendors that have built products on net/open/freeBSD. Dragonfly is the one distro I've never once run into in the wild.

gjs278 5y ago
I run garyshood.com on it
nix23 5y ago
I used it for some time ~3y as a backupserver and was happy with it, just test it out.
enriquto 5y ago
I have a Dragonfly install as part of my "pet" test farm. The build environment is similar to the other BSD, but from time to time it triggers strange portability bugs in my codes that I love to fix.

truth_seeker 5y ago

Excuse me what ? If this is really true, why isn't it more widely adopted than Ubuntu for server side deployment ? then following text is copied from "features" page of official site >EXTREME SCALING DragonFly will autotune kernel resources and scaling metrics such as kernel hash-tables based on available memory. The autoscaling has reached a point where essentially all kernel components will scale to extreme levels. Process and thread components now scale to at least a million user processes or threads, given sufficient physical memory to support that much (around 128GB minimum for one million processes). The PID is currently limited to 6 digits, so discrete user processes are capped at one million, but the (process x thread) matrix can conceivably go much higher. Process creation, basic operation, and destruction have been tested to 900,000 discrete user processes. File data caching scales indefinitely, based on available memory. A very generous kern.maxvnodes default allows the kernel to scale up to tracking millions of files for caching purposes. IPI signaling between CPUs has been heavily optimized and will scale nicely up to the maximum hardware thread limit (256 cpu threads, typically in a 128-core/256-thread configuration). Unnecessary IPIs are optimized out, and the signaling of idle cpus can be further optimized via sysctl parameters. All major kernel resource components are fully SMP-aware and use SMP-friendly algorithms. This means that regular UNIX operations that manipulate PIDs, GIDs, SSIDs, process operations, VM page faults, memory allocation and freeing, pmap updates, VM page sharing, the name cache, most common file operations, process sleep and wakeup, and locks, are all heavily optimized and scale to systems with many cpu cores. In many cases, concurrent functions operate with no locking conflicts or contention. The network subsystem was rewritten pretty much from the ground-up to fully incorporate packet hashes into the entire stack, allowing connections and network interfaces to operate across available CPUs concurrently with little to no contention. Pipes and Sockets have also been heavily optimized for SMP operation. Given a machine with sufficient capability, hundreds of thousands of concurrent TCP sockets can operate efficiently and packet routing capabilities are very high. The disk subsystem, particularly AHCI (SATA) and NVMe, are very SMP friendly. NVMe, in particular, will configure enough hardware queues such that it can dispatch requests and handle responses on multiple cpus simultaneously with no contention. The scheduler uses per-cpu algorithms and scales across any number of cpus. In addition, the scheduler is topology-aware and gains hints from whatever IPC (Inter-Process Communications) occurs to organize active processes within the cpu topology in a way that makes maximum use of cache locality. Load is also taken into account, and can shift how cache locality is handled. The kernel memory manager is somewhat NUMA aware. Most per-cpu operations use NUMA-local memory allocations. User memory requests are also NUMA aware, at least for short-lived user programs. Generally speaking, the scheduler will try to keep a process on the same cpu socket but ultimately we've determined that load balancing is sometimes more important. CPU caches generally do a very good job of maximizing IPC (Instructions Per Clock). Because memory management is fully SMP-aware, a multi-core system can literally allocate and free memory at a rate in the multiple gigabytes/sec range. Generally very high concurrency with very low kernel overhead. The kernel can handle just about any load thrown at it and still be completely responsive to other incidental tasks. Systems can run efficiently at well over 100% load. Supports up to 4 swap devices for paging and up to 55TB (Terabytes) of configured swapspace. Requires 1MB of physical ram per 1GB of configured swap. When multiple swap devices are present, I/O will be interleaved for maximum effectiveness. The paging system is extremely capable under virtually any load conditions, particularly when swap is assigned to NVMe storage. Concurrent page-in across available cpus, in particular, works extremely well. Asynchronous page-out. Extended filesystem data caching via the swapcache mechanism can operate as an extended (huge) disk cache if desired, and/or used to increase the apparent total system memory.

mikece 5y ago

Dragonfly does not support ARM64 (aarch64) but only x86-64???

pikrzyszto 5y ago
yes - one of the reasons behind Dragonfly conception was that it's hard to write high-performant code for all architectures at the same time. Ditching ARM64 and others means much cleaner codebase and less effort to maintain all of this.
- webaholic 5y ago
  If portability of other OSes has taught us anything, it is that the interfaces are much cleaner if you are to support multiple architectures. I think it comes down to having people/companies interested in porting it to ARM64, not about cleaner codebases.
  - nix23 5y ago
    You are absolutely correct, however DragonFly focused since the beginning on x86, that was the target platform for a cluster and the architecture every dev had. Maybe it's time to rethink that.

ncmncm 5y ago

Still validating with md5, in 2021? DragonflyBSD distinguishes itself by supporting a variant of the Apollo Aegis, and later Domainix, practice of expanding environment variable names in symbolic link text. Of course it is more clumsy than in Aegis, for the sake of compatibility with other BSDs and systems that might, by long odds, happen to have what look like environment variable references in their symbolic links, but aren't.