3 ms·

Postgres is one of the few projects that still use a strict patch-oriented development process that's based almost entirely around mailing-list communication.

Postgres is one of the few projects that still use a strict patch-oriented development process that's based almost entirely around mailing-list communication. While core team members can commit directly to the repo, everyone else must submit the code changes for review to the pgsql-hackers mailing list as a clean, self-contained patch, where it's discussed and considered for inclusion. An accepted patch might be committed right away, or it will be queued up for the next scheduled "commitfest" [2], when patches are reviewed and finally committed to mainline. (I don't know how the commitfest interacts with git exactly; the commitfest database doesn't even link to git, only to email discussions.) From the outside it seems a bit antiquated, but it's apparently been working well for them. The Postgres team is a pretty conservative bunch; they only switched from CVS to git in late 2010, for example. They also really care about code quality, getting the design right early, and covering all possible edge cases. As a result, Postgres solid, clean, has unusually few legacy oddities, and almost never any subtle, suprising breaking changes. If you read the MySQL manual, it's absolutely littered with sloppy little breakages throughout its history: Like how, until 5.0.something, when comparing a "date" value with a "datetime" value, the time portion would be silently ignored and ('2017-04-08 14:04' = '2017-04-08') would return true; but they fixed that, and broke a lot of client code because they didn't stop to realize that a lot of developers depended on that behaviour. [1] https://wiki.postgresql.org/wiki/Submitting_a_Patch https://wiki.postgresql.org/wiki/Submitting_a_Patch [2] https://commitfest.postgresql.org https://commitfest.postgresql.org

scrollaway 10y ago
Wine is another such project :) But I gotta say it's only working for those projects because they have an extremely high barrier of entry to the code itself in the first place (working on projects like Wine and Postgres is scary, even though you can get started with easy stuff). It also works for them because they have maintainers and core committers used to the workflow, already tooled on the workflow etc. But I wonder how much productivity would be gained by using a github-like flow maybe enhanced a bit.
- anarazel 10y ago
  In my opinion github isn't quite ready for more complex things: - Lengthy reviews yield PRs that measure many megabytes, become unusably slow - There's no reasonable way to do PRs based on other PRs, without closing and reopening then when the underlying branch is merged - large changes can't even be displayed - doesn't work offline. I do a fair amount of review while traveling, and not being able to do reviews while offline would prevent me from doing so Then there's political issues: there's absolutely no guarantee github will be around in a few years. Postgres is old, and will hopefully live long into three future. When it started there was no sourceforge, and now there's again no sf. I very regularly look back into old development discussions...
  - scrollaway 10y ago
    Yeah absolutely, hence why I said github-like. What I mean by that is something more modern, that supports such review style. If only Github supported mailing lists...
    - pgaddict 10y ago
      There's still the aspect of handing over the control to a third party that may disappear next month, discontinue the features you rely on, etc. FWIW, I don't think the workflow is a major barrier. Do 'git format-patch' and submit it to mailing list is about as simple as it can be. It's all the other stuff (complex code-base, going through review, ...) and fairly high bar for quality that makes it challenging.
pgaddict 10y ago
Linux kernel is another such project, I think. One of the reasons why it's done this way (through mailing lists and not e.g. through pull requests on github) is that all the history is tracked in a way that's fully under control of the community. So it's fairly easy to find who/when submitted the patch, how it looked like, etc. Of course, another reason is history - most of the process was established long before git, when CVS was the VCS.
js2 10y ago
Git itself is another such project.
lathiat 10y ago
> If you read the MySQL manual, it's absolutely littered with sloppy little breakages throughout its history: Like how, until 5.0.something, when comparing a "date" value with a "datetime" value, the time portion would be silently ignored and ('2017-04-08 14:04' = '2017-04-08') would return true; but they fixed that, and broke a lot of client code because they didn't stop to realize that a lot of developers depended on that behaviour. This is an interesting comment for two reasons. Firstly because a lot of people also complain about MySQL's archaic defaults which often stay too long because of upgrade concerns (though they fortunately are fixing a lot of them already or for MySQL 8.0 - hooray). But also because it speaks volumes, in my opinion, about the MySQL documentation that these are documented in the first place. I worked at MySQL for 9 years and though it was always clear our manual was always a good source of information, now that I am working on Ubuntu & OpenStack it is painfully obvious just how good the MySQL documentation team and processes were compared to many other projects. Even just the version ChangeLog. I'm not saying other projects don't get it right (and have no opinion at all about postgresql's documentation state), but MySQL seems to get it pretty right in general.
- lobster_johnson 10y ago
  But MySQL's problem is that those "archaic defaults" shouldn't have happened in the first place. No matter how good MySQL is good at documenting their bugginess, weirdness, flakiness and overall history of legacy warts, the point was PostgreSQL eliminates pretty much all of this through promoting, at every single stage of the development process, the same strict, uncompromising QA principles. By doing this, legacy behaviour generally disappears. Postgres isn't bug-free, of course. But they take great care to not be continually chased by a tsunami of technical debt. If you build a house on a crappy foundation, you get a crappy house, so it's a good idea to spend time on the foundation before building the house. The Postgres team spent years on the foundation, before building the higher-level parts, and the rewards are obvious. Meanwhile, MySQL has spent years and years slowly mopping up tech debt. Things have gotten a lot better with the tighter semantics, such as preventing "February 29th" from being inserted, or rejecting "0000-00-00" as a date, or silently ignoring data coercion errors, and so on. But those things shouldn't have happened in the first place if the developers had been better at QA. So while you're right that the documentation is decent at describing various legacy semantics, it's also encodes a history of carelessness that's rather embarrassing reading. By the way, date/datetime regression I mention isn't in the documentation at all. It's in their bug tracker.

scrollaway 10y ago

Wine is another such project :) But I gotta say it's only working for those projects because they have an extremely high barrier of entry to the code itself in the first place (working on projects like Wine and Postgres is scary, even though you can get started with easy stuff). It also works for them because they have maintainers and core committers used to the workflow, already tooled on the workflow etc. But I wonder how much productivity would be gained by using a github-like flow maybe enhanced a bit.

anarazel 10y ago
In my opinion github isn't quite ready for more complex things: - Lengthy reviews yield PRs that measure many megabytes, become unusably slow - There's no reasonable way to do PRs based on other PRs, without closing and reopening then when the underlying branch is merged - large changes can't even be displayed - doesn't work offline. I do a fair amount of review while traveling, and not being able to do reviews while offline would prevent me from doing so Then there's political issues: there's absolutely no guarantee github will be around in a few years. Postgres is old, and will hopefully live long into three future. When it started there was no sourceforge, and now there's again no sf. I very regularly look back into old development discussions...
- scrollaway 10y ago
  Yeah absolutely, hence why I said github-like. What I mean by that is something more modern, that supports such review style. If only Github supported mailing lists...
  - pgaddict 10y ago
    There's still the aspect of handing over the control to a third party that may disappear next month, discontinue the features you rely on, etc. FWIW, I don't think the workflow is a major barrier. Do 'git format-patch' and submit it to mailing list is about as simple as it can be. It's all the other stuff (complex code-base, going through review, ...) and fairly high bar for quality that makes it challenging.

pgaddict 10y ago

Linux kernel is another such project, I think. One of the reasons why it's done this way (through mailing lists and not e.g. through pull requests on github) is that all the history is tracked in a way that's fully under control of the community. So it's fairly easy to find who/when submitted the patch, how it looked like, etc. Of course, another reason is history - most of the process was established long before git, when CVS was the VCS.

js2 10y ago

Git itself is another such project.

lathiat 10y ago

> If you read the MySQL manual, it's absolutely littered with sloppy little breakages throughout its history: Like how, until 5.0.something, when comparing a "date" value with a "datetime" value, the time portion would be silently ignored and ('2017-04-08 14:04' = '2017-04-08') would return true; but they fixed that, and broke a lot of client code because they didn't stop to realize that a lot of developers depended on that behaviour. This is an interesting comment for two reasons. Firstly because a lot of people also complain about MySQL's archaic defaults which often stay too long because of upgrade concerns (though they fortunately are fixing a lot of them already or for MySQL 8.0 - hooray). But also because it speaks volumes, in my opinion, about the MySQL documentation that these are documented in the first place. I worked at MySQL for 9 years and though it was always clear our manual was always a good source of information, now that I am working on Ubuntu & OpenStack it is painfully obvious just how good the MySQL documentation team and processes were compared to many other projects. Even just the version ChangeLog. I'm not saying other projects don't get it right (and have no opinion at all about postgresql's documentation state), but MySQL seems to get it pretty right in general.

lobster_johnson 10y ago
But MySQL's problem is that those "archaic defaults" shouldn't have happened in the first place. No matter how good MySQL is good at documenting their bugginess, weirdness, flakiness and overall history of legacy warts, the point was PostgreSQL eliminates pretty much all of this through promoting, at every single stage of the development process, the same strict, uncompromising QA principles. By doing this, legacy behaviour generally disappears. Postgres isn't bug-free, of course. But they take great care to not be continually chased by a tsunami of technical debt. If you build a house on a crappy foundation, you get a crappy house, so it's a good idea to spend time on the foundation before building the house. The Postgres team spent years on the foundation, before building the higher-level parts, and the rewards are obvious. Meanwhile, MySQL has spent years and years slowly mopping up tech debt. Things have gotten a lot better with the tighter semantics, such as preventing "February 29th" from being inserted, or rejecting "0000-00-00" as a date, or silently ignoring data coercion errors, and so on. But those things shouldn't have happened in the first place if the developers had been better at QA. So while you're right that the documentation is decent at describing various legacy semantics, it's also encodes a history of carelessness that's rather embarrassing reading. By the way, date/datetime regression I mention isn't in the documentation at all. It's in their bug tracker.