3 ms·

PostgreSQL with sharding is called Postgres-XC.

by j-kidd 14y ago

taligent 14y ago
Only tested with CentOS 5.4 and is at version 1.0 ? Seems odd for PostgreSQL not to have made cluster features more of a priority given the more towards cloud based deployments.
- mnutt 14y ago
  Postgres's replication is very good these days, and it works for many of the read-scaling use cases. Scaling writes is a harder problem and I don't think there's a automatic drop-in solution for any sql store that I know of. Mongo gets away with it because it's non-relational so you never have to worry about joining across servers.
- j-kidd 14y ago
  Probably because the PostgreSQL people value humility more than hype.
- fdr 14y ago
  Well, PostgreSQL is a project -- not a company. As a project, it is very roughly run by consensus, and that means acting strategically is not something the project as a whole does very much. Sure, there are many project goals, but they're rather informed by what various correspondents of pgsql-hackers (this could be loosely coined "the community". Admission is free.) seem personally willing, interested and/or funded to do. You, too, can be a database internals engineer! And Postgres's code is still considered crisp enough to fork for Your Very Own Database Startup (which I think is pretty remarkable -- most such stand-alone programs are typically not worth reworking), so you can waltz right on in. One piece of project gridlock is that the bar is very, very high for any new code to be committed (and almost certainly released), and that goes double if you are not someone who is proven to suffer through all the bugs one is going to find over the foreseeable future and no one else is excited about doing that for you (if you submitted some excellently written, badly needed functionality, I think it would be accepted under that rationale, though, in spite of most complexity concerns). Postgres has a long history, so "the foreseeable future" lasts an awfully long time. Some old features like the way hash-group and hash indexes work I am reasonably confident could not be committed if submitted as-is today. (The former can't spill to disk and the latter has no crash recovery, which is why the documentation shoos you away...but some people use it?) Besides bugs, there also needs to be a lot of convincing that the feature is worth whatever complexity it brings, in implementation and in user-interface. In spite of that, a lot of code has gotten committed, and not without some trepidation as to the sheer amount of growth. I used to hack a fork of Postgres a few years ago, and the project is probably about 10-20% bigger now: http://www.ohloh.net/p/postgres/analyses/latest/languages_summary http://www.ohloh.net/p/postgres/analyses/latest/languages_su... The problem is hard, there is not even a single solution everyone agrees is the best, reliability and maintenance requirements are stringent, and even in spite of all that the project's code is growing at a rather scary clip. All in all though, I think a lot of the pre-requisites for what you want are locked up in the attack on logical replication, which is ongoing in the 9.3 release cycle.
fdr 14y ago
I think XC is a shared-nothing but globally consistent system. This lets you get some scaling up to a handful of nodes (maybe, charitably, a dozen...or two or three dozen), but as something that gives you ACID it is bound to have contention on the transaction coordinator at some point. I'd say it is closer to Oracle RAC in the space of use cases, without some of the nutty (but very impressive, and expensive) engineering to get this "shared-something" tradeoff RAC has.

PostgreSQL with sharding is called Postgres-XC.

by j-kidd 14y ago

PostgreSQL with sharding is called Postgres-XC.

Only tested with CentOS 5.4 and is at version 1.0 ? Seems odd for PostgreSQL not to have made cluster features more of a priority given the more towards cloud based deployments.

mnutt 14y ago
Postgres's replication is very good these days, and it works for many of the read-scaling use cases. Scaling writes is a harder problem and I don't think there's a automatic drop-in solution for any sql store that I know of. Mongo gets away with it because it's non-relational so you never have to worry about joining across servers.
j-kidd 14y ago
Probably because the PostgreSQL people value humility more than hype.
fdr 14y ago
Well, PostgreSQL is a project -- not a company. As a project, it is very roughly run by consensus, and that means acting strategically is not something the project as a whole does very much. Sure, there are many project goals, but they're rather informed by what various correspondents of pgsql-hackers (this could be loosely coined "the community". Admission is free.) seem personally willing, interested and/or funded to do. You, too, can be a database internals engineer! And Postgres's code is still considered crisp enough to fork for Your Very Own Database Startup (which I think is pretty remarkable -- most such stand-alone programs are typically not worth reworking), so you can waltz right on in. One piece of project gridlock is that the bar is very, very high for any new code to be committed (and almost certainly released), and that goes double if you are not someone who is proven to suffer through all the bugs one is going to find over the foreseeable future and no one else is excited about doing that for you (if you submitted some excellently written, badly needed functionality, I think it would be accepted under that rationale, though, in spite of most complexity concerns). Postgres has a long history, so "the foreseeable future" lasts an awfully long time. Some old features like the way hash-group and hash indexes work I am reasonably confident could not be committed if submitted as-is today. (The former can't spill to disk and the latter has no crash recovery, which is why the documentation shoos you away...but some people use it?) Besides bugs, there also needs to be a lot of convincing that the feature is worth whatever complexity it brings, in implementation and in user-interface. In spite of that, a lot of code has gotten committed, and not without some trepidation as to the sheer amount of growth. I used to hack a fork of Postgres a few years ago, and the project is probably about 10-20% bigger now: http://www.ohloh.net/p/postgres/analyses/latest/languages_summary http://www.ohloh.net/p/postgres/analyses/latest/languages_su... The problem is hard, there is not even a single solution everyone agrees is the best, reliability and maintenance requirements are stringent, and even in spite of all that the project's code is growing at a rather scary clip. All in all though, I think a lot of the pre-requisites for what you want are locked up in the attack on logical replication, which is ongoing in the 9.3 release cycle.

fdr 14y ago

I think XC is a shared-nothing but globally consistent system. This lets you get some scaling up to a handful of nodes (maybe, charitably, a dozen...or two or three dozen), but as something that gives you ACID it is bound to have contention on the transaction coordinator at some point. I'd say it is closer to Oracle RAC in the space of use cases, without some of the nutty (but very impressive, and expensive) engineering to get this "shared-something" tradeoff RAC has.