3 ms·

The link in the article ( http://lists.dragonflybsd.org/pipermail/commits/2018-May/672263.html http://lists.dragonflybsd.org/pipermail/commits/2018-May/672... )

by nayuki 8y ago

The link in the article ( http://lists.dragonflybsd.org/pipermail/commits/2018-May/672263.html http://lists.dragonflybsd.org/pipermail/commits/2018-May/672... ) is more helpful than the article itself. > I am somewhat tickled by the notion that you might have a problem after deleting half a billion directory entries. What's wrong with performing 500 million operations on a file system? There is an expectation of very high reliability when it comes to file systems.

userbinator 8y ago
Especially since it's a race condition, depending on the exact environment someone might never see it, experience it once in 500M operations, or see it every few minutes. Many of the hardest bugs I've seen are related to multithreading. Writing code while being mindful not to create buffer overflows is almost trivial in comparison to writing correct multithreaded code with fine-grained locking. I've experienced many cases in the past where someone thought a certain set of race conditions would "never" happen (as documented in FIXME type comments in the code) --- and it basically always does. "I didn't know another thread could be doing that..." is a common sentiment my coworkers have expressed whenever I find bugs of this nature.
- AboutTheWhisles 8y ago
  When writing multi-threaded programs, data structures, etc. you have to make sure problems can't possibly happen - anything else and it's broken.
taspeotis 8y ago
The git commit says "focus vs modify race" so it sounds like a race condition. I don't understand the "notion" either - to me it reads that the probability is so low it happens for a pair of operations once every 500 million operations or so. If it is a race condition you might have the issue after doing just two operations.
paulmd 8y ago
> What's wrong with performing 500 million operations on a file system? There is an expectation of very high reliability when it comes to file systems. And that's why my take for Hammer2 has been: sounds great, but there's not enough usage to shake out the bugs. DragonflyBSD is a 1% of the BSD community, which is like 5% of the total *nix deploys. ZFS has trillions of hours of drive-time. What does Hammer2 have? A great developer? Everyone knows one great developer isn't enough, there are bugs there and not enough usage to shake them out. He could be Actual Literal Jesus and he would still write a small minority of unpredictable, difficult-to-diagnose bugs that have to be sterilized out of the code. (as a total nobody, on my part - it's just not good enough for even casual usage vs the competition here)
- unixhero 8y ago
  At this stage DragonFlyBSD is a research OS and is not attempting to compete against anyone. It exists and innovates, that's what Matt does here. It is enjoyable to use as well!
- ceratopisan 8y ago
  Huge, multinational corporations with products in every house and business across the world also write a small minority of unpredictable, difficult-to-diagnose bugs. (Meltdown, Spectre, LazyFP, the next shoe that drops in a few days, for easy examples) Pick whatever is more fun to use.
ceratopisan 8y ago
You are misinterpreting the sentence. Most people's exhaustive testing regimen is to try something a few times, shrug, and say "works for me". Not necessarily for file systems, but for a lot of software issues. When I wrote that sentence, I was entertained by the somewhat casual "retried this 500,000 times just to see" attitude.

by nayuki 8y ago

userbinator 8y ago

Especially since it's a race condition, depending on the exact environment someone might never see it, experience it once in 500M operations, or see it every few minutes. Many of the hardest bugs I've seen are related to multithreading. Writing code while being mindful not to create buffer overflows is almost trivial in comparison to writing correct multithreaded code with fine-grained locking. I've experienced many cases in the past where someone thought a certain set of race conditions would "never" happen (as documented in FIXME type comments in the code) --- and it basically always does. "I didn't know another thread could be doing that..." is a common sentiment my coworkers have expressed whenever I find bugs of this nature.

AboutTheWhisles 8y ago
When writing multi-threaded programs, data structures, etc. you have to make sure problems can't possibly happen - anything else and it's broken.

taspeotis 8y ago

The git commit says "focus vs modify race" so it sounds like a race condition. I don't understand the "notion" either - to me it reads that the probability is so low it happens for a pair of operations once every 500 million operations or so. If it is a race condition you might have the issue after doing just two operations.

paulmd 8y ago

> What's wrong with performing 500 million operations on a file system? There is an expectation of very high reliability when it comes to file systems. And that's why my take for Hammer2 has been: sounds great, but there's not enough usage to shake out the bugs. DragonflyBSD is a 1% of the BSD community, which is like 5% of the total *nix deploys. ZFS has trillions of hours of drive-time. What does Hammer2 have? A great developer? Everyone knows one great developer isn't enough, there are bugs there and not enough usage to shake them out. He could be Actual Literal Jesus and he would still write a small minority of unpredictable, difficult-to-diagnose bugs that have to be sterilized out of the code. (as a total nobody, on my part - it's just not good enough for even casual usage vs the competition here)

unixhero 8y ago
At this stage DragonFlyBSD is a research OS and is not attempting to compete against anyone. It exists and innovates, that's what Matt does here. It is enjoyable to use as well!
ceratopisan 8y ago
Huge, multinational corporations with products in every house and business across the world also write a small minority of unpredictable, difficult-to-diagnose bugs. (Meltdown, Spectre, LazyFP, the next shoe that drops in a few days, for easy examples) Pick whatever is more fun to use.

ceratopisan 8y ago

You are misinterpreting the sentence. Most people's exhaustive testing regimen is to try something a few times, shrug, and say "works for me". Not necessarily for file systems, but for a lot of software issues. When I wrote that sentence, I was entertained by the somewhat casual "retried this 500,000 times just to see" attitude.