9 ms·

PostgreSQL: the good, the bad, and the ugly

177 points by pykello 11y ago

ExpiredLink 11y ago
... for PostgreSQL developers, not users.
- nwenzel 11y ago
  But it is a look at the process that eventually impacts users. It explains why upset has taken so long. A bit of chaos and a bit of politics.
  - ProblemFactory 11y ago
    I don't think upsert has taken so long because of the community and processes, but because the Postgresql team has high requirements for it: http://www.depesz.com/2012/06/10/why-is-upsert-so-complicated/ http://www.depesz.com/2012/06/10/why-is-upsert-so-complicate... For example, upsert that is worth releasing should: * Work with all unique indexes that may exist on a table, not only a primary key, * Be correct even at highly concurrent writes, at any isolation level from Read Uncommitted to Serializable, * Be fast even at highly concurrent writes, so no table locks, locks may not be held for more than an instant, and no re-running the entire transaction in a loop, * Never throw an "another transaction modified data" exception at Read Committed isolation level, it should always correctly succeed as an insert or update even if another parallel transaction inserted or deleted the matching row. I'm not familiar much with the other database engines, but my impression has been that their upsert or SQL merge do not provide all of these.
- brlewis 11y ago
  From the "bad" section: Robert Haas asked for suggestions toward the solution of some nasty data-corruption issues associated with the "multixact" feature [...] the relevant multixact changes were merged during the 9.3 development cycle. The 9.3 release happened in September 2013, but the fallout from that particular change is still being dealt with. there is concern within the PostgreSQL community that its well-earned reputation for low bug rates is at risk
nwenzel 11y ago
Nothing quite like seeing the sausage get made. But, the open discussion, while perhaps unnerving, is certainly preferred (IMHO) to a closed source development process happening behind closed doors. A bit of chaos is the price to pay for the open process. Command-and-control does have some advantages. And maybe as Postgres grows, the advantages related to a more orderly process become more visible. (For the record, I think Postgres is a fantastic RDBMS + JSON store)
- arithma 11y ago
  Placing JSON store at equal footing with RDBMS is a little bit loaded.
  - deleted 11y ago
    [deleted]
mr_gaga 11y ago
Article is obviously for PostgreSQL developers BUT my gripe as a user (coming from MySQL).....the hardest part about transitioning from MySQL to PostgreSQL isn't PostgreSQL itself but transitioning from phpMyAdmin to phpPgAdmin. phpPgAdmin seems like it's trying to make my life difficult.
- jperras 11y ago
  Stop using web-based admin tools, and learn how to use the CLI interface. I mean this in the best, nicest possible way; we've all been there. But learning how to use the CLI will pay dividends in the short and long run.
  - njs12345 11y ago
    Or, as a middle ground, JetBrains have a nice SQL IDE: https://www.jetbrains.com/dbe/ https://www.jetbrains.com/dbe/
    - javajosh 11y ago
      Yes, it is very good and works with anything not just PG. Of course pgAdmin is quite good too.
    - jimmar 11y ago
      But not released yet? I clicked the link saw that I could only get it through an early access release program.
      - _petronius 11y ago
        For some reason the email you get sent when you sign up for the early-access program also trips Thunderbird's scam warning feature (in addition to the form being submitted over HTTP, despite the site being served over HTTPS). I wonder if that's because the message contains the string "Hi Friend"?
      - z-e-r-o 11y ago
        Google "jetbrains dbe dmg" and it'll link you their knowledge base article with download links to the latest version. It's seriously a fantastic program!
pjungwir 11y ago
I lurk (mostly) on the Postgres general mailing list (not the hacker list), and I recognize a lot of the names in that article. I've got to say that the helpfulness and civility of the folks there is amazing. I'm sorry to hear there are tensions around the release process. I'm glad the article expressed that in spite of those tensions, there is basically good will. That fits my impression of the community. I don't write much C, but I've got a few features I'd love to find time to contribute. I'd be honored to be a part of that project.
dimino 11y ago
> The experience of Firefox and Chrome is that long release cycles actually decrease quality, precisely because of the dynamic on display here: long release cycles create huge pressure to land under-done features just before a deadline. With date-driven short release cycles, if you're not confident in your feature you'll be happy to just slip it to the next release. It greatly reduces stress on developers in general. This comment really resonates with me, and wasn't something I thought of -- the pressure to deliver when your release cycle is so infrequent is something I hadn't noticed before but once mentioned, realize it's absolutely true.
- hyperpape 11y ago
  In my experience, this is not the case if you have short release cycles combined with hard commitments. I work at a company that does enterprise software that releases every 8 weeks, and there are still problems when we commit to shipping a feature for a particular release.
  - techdragon 11y ago
    The nominal word being commitment. If you commit to the feature you haven't been able to take advantage of the short window to make "letting it slip" a less painful option.
    - hyperpape 11y ago
      I agree.
- mook 11y ago
  Unfortunately, it seems like short release cycles can also lead to buggier released; after all, who cares, it'll all be fixed in eight weeks or whatever, right? It's not like people will be using the broken thing for a year. Of course, the next release will then have a _different_ minor bug. Then the pattern repeats.
  - organsnyder 11y ago
jilted 11y ago
I think the PostgreSQL Development Group does a great job overall. If they indeed have a reputation of being hard to work with, all the better in my opinion if it means guarding the integrity of the Project.
- pyre 11y ago
  > if it means guarding the integrity of the Project That's a big "if." There are many projects that have been controlled by "hard to work with" developers, and it's not always a good thing. Even if the project doesn't suffer significantly from it, that doesn't mean it's helping the project either. That may not be the case here, but just as a general statement, you can't chalk up "hard to work with" as equating to "focused on quality" and/or "good for the project."
  - boomzilla 11y ago
    It's open source with a very liberal license. If someone is too difficult to work with, the rest will just fork and move on with life. Do you think the fact that people stick around, and have a respectful discussions means that the guys are not too hard to work with, and the values they provide outweigh the challenges of working with them?
    - pyre 11y ago
      My comment was more on the general "being hard to work with is a sign of skill" idea that pervades the industry. > If someone is too difficult to work with, the rest will just fork and move on with life. This is a gross simplification.
_jmar777 11y ago
A prime example of handjamming inconsequential happenings into a good/bad/ugly blogpost format. If this is what counts as "the ugly", they're doing pretty dang good.
slashnull 11y ago
If it starts to suck like MySQL, will it become popular like MySQL? I think that'd be a fair tradeoff.

177 points by pykello 11y ago

ExpiredLink 11y ago

... for PostgreSQL developers, not users.

nwenzel 11y ago
But it is a look at the process that eventually impacts users. It explains why upset has taken so long. A bit of chaos and a bit of politics.
- ProblemFactory 11y ago
  I don't think upsert has taken so long because of the community and processes, but because the Postgresql team has high requirements for it: http://www.depesz.com/2012/06/10/why-is-upsert-so-complicated/ http://www.depesz.com/2012/06/10/why-is-upsert-so-complicate... For example, upsert that is worth releasing should: * Work with all unique indexes that may exist on a table, not only a primary key, * Be correct even at highly concurrent writes, at any isolation level from Read Uncommitted to Serializable, * Be fast even at highly concurrent writes, so no table locks, locks may not be held for more than an instant, and no re-running the entire transaction in a loop, * Never throw an "another transaction modified data" exception at Read Committed isolation level, it should always correctly succeed as an insert or update even if another parallel transaction inserted or deleted the matching row. I'm not familiar much with the other database engines, but my impression has been that their upsert or SQL merge do not provide all of these.
brlewis 11y ago
From the "bad" section: Robert Haas asked for suggestions toward the solution of some nasty data-corruption issues associated with the "multixact" feature [...] the relevant multixact changes were merged during the 9.3 development cycle. The 9.3 release happened in September 2013, but the fallout from that particular change is still being dealt with. there is concern within the PostgreSQL community that its well-earned reputation for low bug rates is at risk

nwenzel 11y ago

Nothing quite like seeing the sausage get made. But, the open discussion, while perhaps unnerving, is certainly preferred (IMHO) to a closed source development process happening behind closed doors. A bit of chaos is the price to pay for the open process. Command-and-control does have some advantages. And maybe as Postgres grows, the advantages related to a more orderly process become more visible. (For the record, I think Postgres is a fantastic RDBMS + JSON store)

arithma 11y ago
Placing JSON store at equal footing with RDBMS is a little bit loaded.
- deleted 11y ago
  [deleted]

mr_gaga 11y ago

Article is obviously for PostgreSQL developers BUT my gripe as a user (coming from MySQL).....the hardest part about transitioning from MySQL to PostgreSQL isn't PostgreSQL itself but transitioning from phpMyAdmin to phpPgAdmin. phpPgAdmin seems like it's trying to make my life difficult.

jperras 11y ago
Stop using web-based admin tools, and learn how to use the CLI interface. I mean this in the best, nicest possible way; we've all been there. But learning how to use the CLI will pay dividends in the short and long run.
- njs12345 11y ago
  Or, as a middle ground, JetBrains have a nice SQL IDE: https://www.jetbrains.com/dbe/ https://www.jetbrains.com/dbe/
  - javajosh 11y ago
    Yes, it is very good and works with anything not just PG. Of course pgAdmin is quite good too.
  - jimmar 11y ago
    But not released yet? I clicked the link saw that I could only get it through an early access release program.
    - _petronius 11y ago
      For some reason the email you get sent when you sign up for the early-access program also trips Thunderbird's scam warning feature (in addition to the form being submitted over HTTP, despite the site being served over HTTPS). I wonder if that's because the message contains the string "Hi Friend"?
    - z-e-r-o 11y ago
      Google "jetbrains dbe dmg" and it'll link you their knowledge base article with download links to the latest version. It's seriously a fantastic program!

pjungwir 11y ago

I lurk (mostly) on the Postgres general mailing list (not the hacker list), and I recognize a lot of the names in that article. I've got to say that the helpfulness and civility of the folks there is amazing. I'm sorry to hear there are tensions around the release process. I'm glad the article expressed that in spite of those tensions, there is basically good will. That fits my impression of the community. I don't write much C, but I've got a few features I'd love to find time to contribute. I'd be honored to be a part of that project.

dimino 11y ago

> The experience of Firefox and Chrome is that long release cycles actually decrease quality, precisely because of the dynamic on display here: long release cycles create huge pressure to land under-done features just before a deadline. With date-driven short release cycles, if you're not confident in your feature you'll be happy to just slip it to the next release. It greatly reduces stress on developers in general. This comment really resonates with me, and wasn't something I thought of -- the pressure to deliver when your release cycle is so infrequent is something I hadn't noticed before but once mentioned, realize it's absolutely true.

hyperpape 11y ago
In my experience, this is not the case if you have short release cycles combined with hard commitments. I work at a company that does enterprise software that releases every 8 weeks, and there are still problems when we commit to shipping a feature for a particular release.
- techdragon 11y ago
  The nominal word being commitment. If you commit to the feature you haven't been able to take advantage of the short window to make "letting it slip" a less painful option.
  - hyperpape 11y ago
    I agree.
mook 11y ago
Unfortunately, it seems like short release cycles can also lead to buggier released; after all, who cares, it'll all be fixed in eight weeks or whatever, right? It's not like people will be using the broken thing for a year. Of course, the next release will then have a _different_ minor bug. Then the pattern repeats.
- organsnyder 11y ago

jilted 11y ago

I think the PostgreSQL Development Group does a great job overall. If they indeed have a reputation of being hard to work with, all the better in my opinion if it means guarding the integrity of the Project.

pyre 11y ago
> if it means guarding the integrity of the Project That's a big "if." There are many projects that have been controlled by "hard to work with" developers, and it's not always a good thing. Even if the project doesn't suffer significantly from it, that doesn't mean it's helping the project either. That may not be the case here, but just as a general statement, you can't chalk up "hard to work with" as equating to "focused on quality" and/or "good for the project."
- boomzilla 11y ago
  It's open source with a very liberal license. If someone is too difficult to work with, the rest will just fork and move on with life. Do you think the fact that people stick around, and have a respectful discussions means that the guys are not too hard to work with, and the values they provide outweigh the challenges of working with them?
  - pyre 11y ago
    My comment was more on the general "being hard to work with is a sign of skill" idea that pervades the industry. > If someone is too difficult to work with, the rest will just fork and move on with life. This is a gross simplification.

_jmar777 11y ago

A prime example of handjamming inconsequential happenings into a good/bad/ugly blogpost format. If this is what counts as "the ugly", they're doing pretty dang good.

slashnull 11y ago

If it starts to suck like MySQL, will it become popular like MySQL? I think that'd be a fair tradeoff.