8 ms·

DragonFly BSD 5.2

159 points by joeschmoe3 8y ago

dec0dedab0de 8y ago
I never heard of HAMMER2( or HAMMER) until just now, but a quick search seems awesome. I might have to give bsd another try, its been over a decade.
pmoriarty 8y ago
Are there any plans to bring HAMMER2 in to Linux?
- kalessin 8y ago
  Afaik the whole purpose of DragonflyBSD is HAMMER so no.
  - azinman2 8y ago
    Can you go into more detail please? That sounds interesting. I thought it had to do with concurrency.
    - doctorsher 8y ago
      You're not wrong. Interview from 2004 [0]: "DragonFly split off from FreeBSD-5 over major architectural differences, not anything else. We really do feel that FreeBSD-5 is taking the wrong approach to SMP and building something that is so complex that it will ultimately not be maintainable. We think we have a better way." But their original architectural decisions were done with this goal in mind [1]: "The ultimate goal of the DragonFly project at its inception was to provide native clustering support in the kernel. This type of functionality requires a sophisticated cache management framework for filesystem namespaces, file spaces and VM spaces." It's not difficult to see how HAMMER came to be three to four years later, given their stated design goals. [0] http://www.onlamp.com/pub/a/bsd/2004/07/08/dragonfly_bsd_interview.html http://www.onlamp.com/pub/a/bsd/2004/07/08/dragonfly_bsd_int... [1] https://www.dragonflybsd.org/history/ https://www.dragonflybsd.org/history/
  - doctorsher 8y ago
    Dragonfly also has lightweight kernel threads and some other different architectural decisions. Not just HAMMER. But you are right, it will never reach Linux.
  - xi- 8y ago
    Not at all, originally the whole purpose of DragonFly was a test ground for its scheduler. It moved on to being much more though, refactoring the entire source tree and rewriting parts of the system. The whole purpose of DragonFly is DragonFly.
- _emacsomancer_ 8y ago
  From the discussion I've heard, it would be challenging even to port HAMMER2 to other BSDs, so I wouldn't expect to see it on Linux any time very soon.
niftich 8y ago
Just the other day I voiced [1] a concern on a ZFS thread about not enough modern filesystems [2] shipping as defaults (or being recommended as such), despite some of these filesystems having been around for many years. This is a promising move to the contrary. [1] https://news.ycombinator.com/item?id=16798244 https://news.ycombinator.com/item?id=16798244 [2] in some loose, but recognizable definition of modern, featuring checksumming, CoW, deduplication, snapshots...
- Mister_Snuggles 8y ago
  I'm pretty sure the default is still UFS, but FreeBSD allows you to use ZFS as the root filesystem. I've been using this for a while and it's been incredibly solid. It's a first class feature that's supported by the bootloader and the GENERIC kernel. My experience with it has been uneventful, which is exactly how a filesystem should be.
  - _emacsomancer_ 8y ago
    Doesn't TrueOS default to ZFS?
    - krylon 8y ago
      It did when it was still called PC-BSD. I don't think it has changed.
      - Crontab 8y ago
        It still is. From the homepage of the site: "TrueOS has been using OpenZFS as its exclusive file system for several years, ensuring advanced OpenZFS functionality is heavily tested and 100% production-ready."
    - JdeBP 8y ago
      It did even before FreeBSD did.
- AnIdiotOnTheNet 8y ago
azinman2 8y ago
Are there many users of dragon fly bsd? Are people using it in production, and why/why not?
subway 8y ago
MD5 release checksums? c'mon folks, get with the times...
- ceratopisan 8y ago
  I generated them; what would you prefer, and why? I know MD5 hash collisions are vaguely possible, but I would think they would be extremely unlikely here.
  - contingencies 8y ago
    FWIW Gentoo uses multiple algorithms plus file size in package checks, see https://wiki.gentoo.org/wiki/Repository_format/package/Manifest https://wiki.gentoo.org/wiki/Repository_format/package/Manif... I recently made an argument to add a new one, Streebog, and you can read the discussion at https://bugs.gentoo.org/show_bug.cgi?id=597736 https://bugs.gentoo.org/show_bug.cgi?id=597736
  - subway 8y ago
    There's no vague about it. MD5 collisions have been possible to generate for over a decade now. http://www.win.tue.nl/hashclash/SoftIntCodeSign/ http://www.win.tue.nl/hashclash/SoftIntCodeSign/ A sha256 or sha512 would be more appropriate.
    - tedunangst 8y ago
      If you don't trust the dragonfly team to not produce collisions, you probably shouldn't use the software anyway.
      - subway 8y ago
        After this thread, I don't trust the dragonfly team to tie their shoes. Not a chance I'll be using their software.
      - michaelmrose 8y ago
        The malicious case would be a bad actor distributing phony files that appear to check out because the flaws have been found that would allow someone to do so.
agar 8y ago
With DragonFly's architecture being designed with multiprocessing and clustering in mind, how does it fare (relative to Linux and other BSDs) in modern environments with many-core CPUs, horizontally-scalable architectures, reliance on VM-based workloads, and containerization? I would think it would really shine, but haven't seen comparisons - or much market uptake. Wouldn't significant advantages result in lower costs (e.g., fewer CPUs needed because of higher performance per core due to lower OS overhead) and therefore economics would drive greater use? If it hasn't, why not?
- arca_vorago 8y ago
  I did a little testing about ~4 years ago and found most of the speed increases were in the netstack, but I didn't do any computational testing either. As for your last question, I think largely due to the fact that it's just not mainstream yet.
- gh02t 8y ago
  I'm not an expert on DragonFly, but from what I know it's design differences aren't really that radical. It differs substantially from FreeBSD, but it's not like a fundamental re-imagining optimized for SMP. Performance is a bit better in some places, and about the same in others. I do think DragonFly is neat and quite interesting, but I suspect it didn't gain as much momentum because the community behind BSD is already small and reluctant to fracture without stronger reasons. It's worth noting that this is not like spinning off a new Linux distro where you're ultimately just repackaging different variations of the same basic OS - DragonFly is effectively an entirely different OS since the kernel is significantly different. Slightly out of date, but still relevant benchmarks: https://www.phoronix.com/scan.php?page=article&item=freebsd11-beta-benchmarks&num=1 https://www.phoronix.com/scan.php?page=article&item=freebsd1...
  - agar 8y ago
    Thanks, I took a look through that and most (all?) of the benchmarks seemed to be fairly traditional tasks, some not even highly threaded. I guess I'm wondering if complex, multi-box, cloud-scale workloads have any advantage on Dragonfly - or if the optimizations for clustering are in such a narrow band of use cases that modern architectures have passed it by just through brute force. There's an irony here, in that Matt Dillon was an ex-Amiga hacker (IIRC, some of Dragonfly's architectural choices were inspired by the Amiga). Anyway, the Amiga's linear memory, co-processor-driven architecture was ridiculously more elegant than the PC's segmented memory, CPU-bound approach. Didn't matter though, a huge investment in working around the PC's limitations led to its market dominance. So, if massive technical accommodations for less efficient OS architectures end up marginalizing Dragonfly, it would be a bit of history repeating itself. And I'm sure the irony wouldn't be lost on Mr. Dillon.
jasonkostempski 8y ago
A few years ago I wanted to try this as a local Minecraft server but it was 64 bit only and the spare machine I had was 32 bit. Glad this post came up because now I've got a 64 bit machine sitting in my basement I had forgotten about until now.
- h1d 8y ago
  Can you not just spin up a cloud instance and boot it off of a cd image?
gaius 8y ago
Still haven’t quite managed to get Dragonfly going in Azure, but not given up on it either
0xcde4c3db 8y ago
Anyone know what the state of GPU support actually is? The Supported Hardware page [1] only says "Intel GPUs are supported up to the Skylake generation (2016 machines)", but this seems to be outdated. At the very least, Matt Dillon himself reported good OpenGL performance with a Kaby Lake machine [2]. [1] https://www.dragonflybsd.org/docs/supportedhardware/ https://www.dragonflybsd.org/docs/supportedhardware/ [2] http://lists.dragonflybsd.org/pipermail/users/2017-September/313586.html http://lists.dragonflybsd.org/pipermail/users/2017-September...
- ftigeot 8y ago
  DragonFly web documentation is wiki-based and tends to not always be up-to-date (contrary to man pages). All recent Intel GPUs are supported, including Kabylake and Coffeelake.

159 points by joeschmoe3 8y ago

I never heard of HAMMER2( or HAMMER) until just now, but a quick search seems awesome. I might have to give bsd another try, its been over a decade.

pmoriarty 8y ago

Are there any plans to bring HAMMER2 in to Linux?

kalessin 8y ago
Afaik the whole purpose of DragonflyBSD is HAMMER so no.
- azinman2 8y ago
  Can you go into more detail please? That sounds interesting. I thought it had to do with concurrency.
  - doctorsher 8y ago
    You're not wrong. Interview from 2004 [0]: "DragonFly split off from FreeBSD-5 over major architectural differences, not anything else. We really do feel that FreeBSD-5 is taking the wrong approach to SMP and building something that is so complex that it will ultimately not be maintainable. We think we have a better way." But their original architectural decisions were done with this goal in mind [1]: "The ultimate goal of the DragonFly project at its inception was to provide native clustering support in the kernel. This type of functionality requires a sophisticated cache management framework for filesystem namespaces, file spaces and VM spaces." It's not difficult to see how HAMMER came to be three to four years later, given their stated design goals. [0] http://www.onlamp.com/pub/a/bsd/2004/07/08/dragonfly_bsd_interview.html http://www.onlamp.com/pub/a/bsd/2004/07/08/dragonfly_bsd_int... [1] https://www.dragonflybsd.org/history/ https://www.dragonflybsd.org/history/
- doctorsher 8y ago
  Dragonfly also has lightweight kernel threads and some other different architectural decisions. Not just HAMMER. But you are right, it will never reach Linux.
- xi- 8y ago
  Not at all, originally the whole purpose of DragonFly was a test ground for its scheduler. It moved on to being much more though, refactoring the entire source tree and rewriting parts of the system. The whole purpose of DragonFly is DragonFly.
_emacsomancer_ 8y ago
From the discussion I've heard, it would be challenging even to port HAMMER2 to other BSDs, so I wouldn't expect to see it on Linux any time very soon.

niftich 8y ago

Just the other day I voiced [1] a concern on a ZFS thread about not enough modern filesystems [2] shipping as defaults (or being recommended as such), despite some of these filesystems having been around for many years. This is a promising move to the contrary. [1] https://news.ycombinator.com/item?id=16798244 https://news.ycombinator.com/item?id=16798244 [2] in some loose, but recognizable definition of modern, featuring checksumming, CoW, deduplication, snapshots...

Mister_Snuggles 8y ago
I'm pretty sure the default is still UFS, but FreeBSD allows you to use ZFS as the root filesystem. I've been using this for a while and it's been incredibly solid. It's a first class feature that's supported by the bootloader and the GENERIC kernel. My experience with it has been uneventful, which is exactly how a filesystem should be.
- _emacsomancer_ 8y ago
  Doesn't TrueOS default to ZFS?
  - krylon 8y ago
    It did when it was still called PC-BSD. I don't think it has changed.
    - Crontab 8y ago
      It still is. From the homepage of the site: "TrueOS has been using OpenZFS as its exclusive file system for several years, ensuring advanced OpenZFS functionality is heavily tested and 100% production-ready."
  - JdeBP 8y ago
    It did even before FreeBSD did.
AnIdiotOnTheNet 8y ago

azinman2 8y ago

Are there many users of dragon fly bsd? Are people using it in production, and why/why not?

subway 8y ago

MD5 release checksums? c'mon folks, get with the times...

ceratopisan 8y ago
I generated them; what would you prefer, and why? I know MD5 hash collisions are vaguely possible, but I would think they would be extremely unlikely here.
- contingencies 8y ago
  FWIW Gentoo uses multiple algorithms plus file size in package checks, see https://wiki.gentoo.org/wiki/Repository_format/package/Manifest https://wiki.gentoo.org/wiki/Repository_format/package/Manif... I recently made an argument to add a new one, Streebog, and you can read the discussion at https://bugs.gentoo.org/show_bug.cgi?id=597736 https://bugs.gentoo.org/show_bug.cgi?id=597736
- subway 8y ago
  There's no vague about it. MD5 collisions have been possible to generate for over a decade now. http://www.win.tue.nl/hashclash/SoftIntCodeSign/ http://www.win.tue.nl/hashclash/SoftIntCodeSign/ A sha256 or sha512 would be more appropriate.
  - tedunangst 8y ago
    If you don't trust the dragonfly team to not produce collisions, you probably shouldn't use the software anyway.
    - subway 8y ago
      After this thread, I don't trust the dragonfly team to tie their shoes. Not a chance I'll be using their software.
    - michaelmrose 8y ago
      The malicious case would be a bad actor distributing phony files that appear to check out because the flaws have been found that would allow someone to do so.

agar 8y ago

With DragonFly's architecture being designed with multiprocessing and clustering in mind, how does it fare (relative to Linux and other BSDs) in modern environments with many-core CPUs, horizontally-scalable architectures, reliance on VM-based workloads, and containerization? I would think it would really shine, but haven't seen comparisons - or much market uptake. Wouldn't significant advantages result in lower costs (e.g., fewer CPUs needed because of higher performance per core due to lower OS overhead) and therefore economics would drive greater use? If it hasn't, why not?

arca_vorago 8y ago
I did a little testing about ~4 years ago and found most of the speed increases were in the netstack, but I didn't do any computational testing either. As for your last question, I think largely due to the fact that it's just not mainstream yet.
gh02t 8y ago
I'm not an expert on DragonFly, but from what I know it's design differences aren't really that radical. It differs substantially from FreeBSD, but it's not like a fundamental re-imagining optimized for SMP. Performance is a bit better in some places, and about the same in others. I do think DragonFly is neat and quite interesting, but I suspect it didn't gain as much momentum because the community behind BSD is already small and reluctant to fracture without stronger reasons. It's worth noting that this is not like spinning off a new Linux distro where you're ultimately just repackaging different variations of the same basic OS - DragonFly is effectively an entirely different OS since the kernel is significantly different. Slightly out of date, but still relevant benchmarks: https://www.phoronix.com/scan.php?page=article&item=freebsd11-beta-benchmarks&num=1 https://www.phoronix.com/scan.php?page=article&item=freebsd1...
- agar 8y ago
  Thanks, I took a look through that and most (all?) of the benchmarks seemed to be fairly traditional tasks, some not even highly threaded. I guess I'm wondering if complex, multi-box, cloud-scale workloads have any advantage on Dragonfly - or if the optimizations for clustering are in such a narrow band of use cases that modern architectures have passed it by just through brute force. There's an irony here, in that Matt Dillon was an ex-Amiga hacker (IIRC, some of Dragonfly's architectural choices were inspired by the Amiga). Anyway, the Amiga's linear memory, co-processor-driven architecture was ridiculously more elegant than the PC's segmented memory, CPU-bound approach. Didn't matter though, a huge investment in working around the PC's limitations led to its market dominance. So, if massive technical accommodations for less efficient OS architectures end up marginalizing Dragonfly, it would be a bit of history repeating itself. And I'm sure the irony wouldn't be lost on Mr. Dillon.

jasonkostempski 8y ago

A few years ago I wanted to try this as a local Minecraft server but it was 64 bit only and the spare machine I had was 32 bit. Glad this post came up because now I've got a 64 bit machine sitting in my basement I had forgotten about until now.

h1d 8y ago
Can you not just spin up a cloud instance and boot it off of a cd image?

gaius 8y ago

Still haven’t quite managed to get Dragonfly going in Azure, but not given up on it either

0xcde4c3db 8y ago

Anyone know what the state of GPU support actually is? The Supported Hardware page [1] only says "Intel GPUs are supported up to the Skylake generation (2016 machines)", but this seems to be outdated. At the very least, Matt Dillon himself reported good OpenGL performance with a Kaby Lake machine [2]. [1] https://www.dragonflybsd.org/docs/supportedhardware/ https://www.dragonflybsd.org/docs/supportedhardware/ [2] http://lists.dragonflybsd.org/pipermail/users/2017-September/313586.html http://lists.dragonflybsd.org/pipermail/users/2017-September...

ftigeot 8y ago
DragonFly web documentation is wiki-based and tends to not always be up-to-date (contrary to man pages). All recent Intel GPUs are supported, including Kabylake and Coffeelake.