5 ms·

The meta point here is that just in OSS folks assume that the code has been reviewed since it's open, but the reality is that unless someone actually does you r

The meta point here is that just in OSS folks assume that the code has been reviewed since it's open, but the reality is that unless someone actually does you really don't know. The popularity of a project doesn't mean there aren't fatal flaws.

fxfan 8y ago
This is a contrarian view and I will sound phobic but for this very reason- false sense of security + possibility of malicious check ins- I now place less emphasis on open/closed source and more on repute. I am happier to download exe's from good companies these days than a package from arch-aur
ascar 8y ago
Personally I assume if a project is at least somewhat popular it works most of the time and frequent/serious bugs are reported and researchable. For that to work I also report or second bugs I encounter. That doesn't mean there aren't any bugs in even the most popular OSS, especially in edge cases and rare scenarios. With closed source though I often don't know the popularty and how many/what kind of bugs have been reported, but just the reputation of the vendor. I prefer a popular software from a highly reputable vendor over a somewhat popular OSS. But I also prefer the most popular and battletested OSS, like postgresql and linux, over any closed system, e.g. SQL Server and Windows.
loeg 8y ago
No need to bring out that dead horse for a beating. No one in this thread has yet made the claim that OSS code is flawless; you're picking a fight with a straw man.
- zapzupnz 8y ago
  It's a genuine issue, and one that is made time and again by people who think "open == security" whenever there's a discussion about something like Google or iMessage, when the armchair security experts come out of the woodwork to promote their favourite whatever-it-is. Sure, it mightn't be made in this thread yet, but that doesn't make it an irrelevant, invalid, or uninteresting observation. I think that the spirit of discussion, so integral to what separates HN from other websites, means we should not poo-poo this line of inquiry just because you're bored of it.
  - feanaro 8y ago
    I think the point is much often closer to "open is a prerequisite to security".
    - kelnos 8y ago
      It's more like "open is a prerequisite for personal verification of security". A system can be closed and secure, just you can't verify it.
      - feanaro 8y ago
        Yes, but I tend to view security as a somewhat epistemological phenomenon. It's not enough for the security to exist "somewhere out there in the universe" in an absolute, objective sense. If you have no way of verifying it, it could simply be a lie, and is thus useless for threat modelling.
  - loeg 8y ago
    > Sure, it mightn't be made in this thread yet, but that doesn't make it an irrelevant, invalid, or uninteresting observation. I really think it does. It's like "the sky isn't green!" or "the earth isn't flat!" or "vaccines don't cause autism!" Sure, these are all true things, but they weren't exactly topics of discussion on this thread before you brought them up. By all means, discuss the article, and rebut comments you feel espouse an inaccurate worldview. (IMO) preemptive rebuttals like this are only useful or interesting when they're somewhat novel, or represent some special insight into a particular field that outsiders wouldn't have. This one has neither. My particular take on why this dead horse is irrelevant (as well as tedious and boring): Fsync isn't a security issue, it's a data loss issue. Arguably, the Postgres behavior is quite reasonable and the article's headline is just inaccurate. Linux has been reviewed, e.g., https://danluu.com/file-consistency/ https://danluu.com/file-consistency/ from 2017, summarizing research from 2001-2014, all of which pointed towards deficiencies in its data preservation behavior. The Linux community know they lose data and propose that users should accept it.[0] The Postgres <-> Linux fsync investigation has been ongoing for a long time, with lots of eyeballs on both sides of the kernel/userspace boundary. This isn't really a "bug escapes major application developers for 20 years!" so much as "Linux can't agree to provide an API to make file data consistent." [0]: https://lwn.net/Articles/752105/ https://lwn.net/Articles/752105/ [1]: http://rhaas.blogspot.com/2014/03/linuxs-fsync-woes-are-getting-some.html http://rhaas.blogspot.com/2014/03/linuxs-fsync-woes-are-gett...
- dis-sys 8y ago
  No one is beating a dead horse here, there is a valid point - "The popularity of a project doesn't mean there aren't fatal flaws"
  - loeg 8y ago
    The point is valid, but uninteresting. It's the default. Everyone who has skimmed HN for more than two weeks has seen this by example, if not by comment, time and time again.
- clhodapp 8y ago
  Humorously, someone in this thread has now actually made the claim that their open source database product has flawless crash reliability: https://news.ycombinator.com/item?id=19127011 https://news.ycombinator.com/item?id=19127011
  - ben0x539 8y ago
    but not that it's because it's free software, rather that it's because they read the docs.
  - loeg 8y ago
    Then make whatever appropriate response to that comment you feel is necessary; it still isn't a reasonable top-level comment.
  - hyc_symas 8y ago
    Multiple independent research teams with their crash analysis/fuzzing tools confirmed the fact. Along with over 7 years deployment in large enterprises: Zero crash-induced corruption, zero startup/recovery time. Crash-proof by design.
rleigh 8y ago
Any project can contain fatal flaws irrespective of its review policies. Particularly when we are concerned with the subtle behaviour of the interaction between write and fsync. Even if you read the standard it's not clear exactly how the system as a whole should behave; there are a number of situations which aren't mentioned in the standard at all. It would be quite possible for a review, and even extensive testing, to fail to pick up on some system-specific subtleties.

fxfan 8y ago

This is a contrarian view and I will sound phobic but for this very reason- false sense of security + possibility of malicious check ins- I now place less emphasis on open/closed source and more on repute. I am happier to download exe's from good companies these days than a package from arch-aur

ascar 8y ago

Personally I assume if a project is at least somewhat popular it works most of the time and frequent/serious bugs are reported and researchable. For that to work I also report or second bugs I encounter. That doesn't mean there aren't any bugs in even the most popular OSS, especially in edge cases and rare scenarios. With closed source though I often don't know the popularty and how many/what kind of bugs have been reported, but just the reputation of the vendor. I prefer a popular software from a highly reputable vendor over a somewhat popular OSS. But I also prefer the most popular and battletested OSS, like postgresql and linux, over any closed system, e.g. SQL Server and Windows.

loeg 8y ago

No need to bring out that dead horse for a beating. No one in this thread has yet made the claim that OSS code is flawless; you're picking a fight with a straw man.

zapzupnz 8y ago
It's a genuine issue, and one that is made time and again by people who think "open == security" whenever there's a discussion about something like Google or iMessage, when the armchair security experts come out of the woodwork to promote their favourite whatever-it-is. Sure, it mightn't be made in this thread yet, but that doesn't make it an irrelevant, invalid, or uninteresting observation. I think that the spirit of discussion, so integral to what separates HN from other websites, means we should not poo-poo this line of inquiry just because you're bored of it.
- feanaro 8y ago
  I think the point is much often closer to "open is a prerequisite to security".
  - kelnos 8y ago
    It's more like "open is a prerequisite for personal verification of security". A system can be closed and secure, just you can't verify it.
    - feanaro 8y ago
      Yes, but I tend to view security as a somewhat epistemological phenomenon. It's not enough for the security to exist "somewhere out there in the universe" in an absolute, objective sense. If you have no way of verifying it, it could simply be a lie, and is thus useless for threat modelling.
- loeg 8y ago
  > Sure, it mightn't be made in this thread yet, but that doesn't make it an irrelevant, invalid, or uninteresting observation. I really think it does. It's like "the sky isn't green!" or "the earth isn't flat!" or "vaccines don't cause autism!" Sure, these are all true things, but they weren't exactly topics of discussion on this thread before you brought them up. By all means, discuss the article, and rebut comments you feel espouse an inaccurate worldview. (IMO) preemptive rebuttals like this are only useful or interesting when they're somewhat novel, or represent some special insight into a particular field that outsiders wouldn't have. This one has neither. My particular take on why this dead horse is irrelevant (as well as tedious and boring): Fsync isn't a security issue, it's a data loss issue. Arguably, the Postgres behavior is quite reasonable and the article's headline is just inaccurate. Linux has been reviewed, e.g., https://danluu.com/file-consistency/ https://danluu.com/file-consistency/ from 2017, summarizing research from 2001-2014, all of which pointed towards deficiencies in its data preservation behavior. The Linux community know they lose data and propose that users should accept it.[0] The Postgres <-> Linux fsync investigation has been ongoing for a long time, with lots of eyeballs on both sides of the kernel/userspace boundary. This isn't really a "bug escapes major application developers for 20 years!" so much as "Linux can't agree to provide an API to make file data consistent." [0]: https://lwn.net/Articles/752105/ https://lwn.net/Articles/752105/ [1]: http://rhaas.blogspot.com/2014/03/linuxs-fsync-woes-are-getting-some.html http://rhaas.blogspot.com/2014/03/linuxs-fsync-woes-are-gett...
dis-sys 8y ago
No one is beating a dead horse here, there is a valid point - "The popularity of a project doesn't mean there aren't fatal flaws"
- loeg 8y ago
  The point is valid, but uninteresting. It's the default. Everyone who has skimmed HN for more than two weeks has seen this by example, if not by comment, time and time again.
clhodapp 8y ago
Humorously, someone in this thread has now actually made the claim that their open source database product has flawless crash reliability: https://news.ycombinator.com/item?id=19127011 https://news.ycombinator.com/item?id=19127011
- ben0x539 8y ago
  but not that it's because it's free software, rather that it's because they read the docs.
- loeg 8y ago
  Then make whatever appropriate response to that comment you feel is necessary; it still isn't a reasonable top-level comment.
- hyc_symas 8y ago
  Multiple independent research teams with their crash analysis/fuzzing tools confirmed the fact. Along with over 7 years deployment in large enterprises: Zero crash-induced corruption, zero startup/recovery time. Crash-proof by design.

rleigh 8y ago

Any project can contain fatal flaws irrespective of its review policies. Particularly when we are concerned with the subtle behaviour of the interaction between write and fsync. Even if you read the standard it's not clear exactly how the system as a whole should behave; there are a number of situations which aren't mentioned in the standard at all. It would be quite possible for a review, and even extensive testing, to fail to pick up on some system-specific subtleties.