10 ms·
PolarDB, yet another open source database system based on PostgreSQL
- qaq 5y agoThis field is getting crowded wonder if there will be some type of consolidation.
- sail0rm00n 5y agoThere’s no need for consolidation when Postgres already exists.
- speedgoose 5y agoSometimes I dream about a highly available multi primary postgresql.
- cultofmetatron 5y agoyea, some sort of vitessdb equivalent to postgresql would be amazing!
- supergirl 5y agothere used to be citus but now it's part of microsoft Azure
- mslot 5y agoTrue, it is developed by Microsoft and available as a service on Azure. It is also open source, actively maintained and improved, and it's a PG13-compatible Postgres extension that adds both distributed database capabilities and columnar storage. :) https://github.com/citusdata/citus https://github.com/citusdata/citus (Citus engineer)
- jhgb 5y ago> PG13-compatible So not (IBM-System-)R-compatible, then?
- DaiPlusPlus 5y agoCockroachDB?
- speedgoose 5y agoPerhaps. It's missing the 30 years of experience, the PostGreSQL reputation, an opensource license, and a better name.
- qaq 5y agoYugobyteDB ?
- DaiPlusPlus 5y ago> a better name NGL, I've had to explain to enough people by now that the "funny name" is because it was made by ex-Googlers with a view towards resiliency, as per the idiom that only cockroaches will survive global nuclear war.
- bob1029 5y agoDo you also dream of loss of certain ACID semantics or crippling performance issues? In the real world, it is extremely hard to provide all the same guarantees you get out of a single instance of <database vendor> if you turn around and spread it across the internet. If you have super deep control over the physical & temporal environment around your system, you can cheat the rules a little bit (i.e. Google).
- speedgoose 5y agoYes I know that the perfect database cannot exist. I think some trade-off are possible though. Today I use postgresql with a primary and replicas, together with couchdb in the same application. They complement each other, but I think something good between is possible.
- rad_gruchalski 5y agoI would recommend having a look at Yugabyte.
- zinclozenge 5y agoWhy yugabyte over cockroachdb?
- rad_gruchalski 5y agoLicensing. CockroachDB does not allow as-a-service.
- zinclozenge 5y agoYugabyte does allow it though? I could take it and create my own company based around providing yugabyte as a service?
- rad_gruchalski 5y agoYes. The core database is fully Apache 2 licensed with no strings attached: https://docs.yugabyte.com/latest/legal/ https://docs.yugabyte.com/latest/legal/. Maybe it will change if the company behind feels they're ripped off. Who knows, but for now, yes, you could. CockroachDB has an explicit clause is the licensing: Yes, employees and contractors can use your internal CockroachDB instance as a service, but no people outside of your organization will be able to use it without purchasing a license: https://www.cockroachlabs.com/docs/v21.1/licensing-faqs.html#can-i-host-cockroachdb-as-a-service-for-internal-use-at-my-organization https://www.cockroachlabs.com/docs/v21.1/licensing-faqs.html....
- speedgoose 5y agoThanks. I should try it out.
- qaq 5y agowell some use cases need something that is distributed like PolarDB, YugobyteDB, CockroachDB etc.
- spicybright 5y agoFantastic! I was actually looking for a buggier version of postgres that won't be supported a year. That's exactly what I was looking for. > PolarDB will evolve and offer its functions and features in two major parts: Oh boy.
- deleted 5y ago[deleted]
- edmundsauto 5y agoHow do you think the next dominant Database platform will start? Is your expectation that innovation requires polish before hitting the (free open source) “marketplace”?
- DaiPlusPlus 5y agoMy expectation is that the next dominant platform will offer something unique which creates enough value that I'm willing to overlook the unfinished parts that deduct value. When Neo4j and other graph databases went big in the past decade, they value they created: "finally! I don't have to faff around with RDBMS tables and Cobb purists to store and query my non-relational object-graph!" - despite the fact that Neo4j then (and still?) didn't support running serving databases concurrently or schema enforcement or even transactional atomicity (I think they fixed that recently?) So in this context, what business-value does PolarDB add or create that makes it worthwhile to deal with its expected short-life?
- tzumby 5y ago“high availability through Paxos based replication”. I thought Paxos is supposed to be a CP system.
- jlokier 5y agoYou have N >= 3 nodes, or N >= 2 and a non-compute arbitrator. One of them goes down, stops responding. You still have quorum, data processes continue just fine. That's high-availability in a CP system.
- redis_mlc 5y agoNo database works well across network partitions for general use (you can designate one region for active business transactions, but that means the other regions are effectively offline during the partition.) If you understand your use cases and your application source code, you can go a long way towards some kind of multi-region HA that does something acceptable during the partition, but that's a pretty big if. Source: DBA.
- chousuke 5y agoHA isn't really about 100% availability. Any single system that promises such is extremely likely to be misleading you somehow. Your in-flight query is going to get interrupted no matter how fancy your clustering is, and I struggle to even come up with hypothetical use cases where this is something you can't afford to have happen, ever. All you need is that in the event of a failure the clustered system can still recover quickly enough (to a well-defined state!) that the application layer can deal with the transient failure without significant impact on users, maintaining the illusion of availability.
- mahkeiro 5y agoIt seems that Alibaba is going more open source than Amazon for their clouds services. That’s interesting and they may use this approach as a key differentiator.
- deleted 5y ago[deleted]
- unityByFreedom 5y agoBased on what metric? Amazon took over Elastic's open source server and another major one IIRC. That was huge. PostgreSQL is not about to disappear.
- greatpatton 5y agoThey did it only after ES decided to abandon the Apache license. I doubt that Amazon would have done it if they were not forced. Redshift is also Postgres based and Amazon never released something. So based on that metric Alibaba is more open.
- unityByFreedom 5y agoThere would be no open source Elastic had Amazon not taken it on. It was a smart move and benefits their balance sheet to do so. The point though is that move was greater than replicating existing open source work.
- funny_falcon 5y ago? Team on my previous workplace patched ElasticSearch to add functionality for years. Could thay do it if there were no open source Elastic?
- unityByFreedom 5y agoI agree you can't patch software without an actively supported open source package. You may have meant to reply to someone else.
- wanghq 5y agoOh, what's going on here? > OceanBase, the database of Alibaba's fintech company Ant Group, will be open-source soon, possibly as early as June 1, according to Sina Tech. https://cntechpost.com/2021/05/27/ant-groups-in-house-database-oceanbase-said-to-be-open-source-soon/ https://cntechpost.com/2021/05/27/ant-groups-in-house-databa... Also the team published many papers about PolarDB. https://scholar.google.com/scholar?hl=en&as_sdt=0%2C48&q=%22polardb%22&btnG= https://scholar.google.com/scholar?hl=en&as_sdt=0%2C48&q=%22...
- wejick 5y agoLooks like they're different technologies, found that oceandb has been offered in AliCloud under ApsaraDB umbrella https://www.alibabacloud.com/product/oceanbase https://www.alibabacloud.com/product/oceanbase
- foobarbazetc 5y agoAlibaba is huge and different teams have different goals. That one is for fintech.
- yftsui 5y agoThese are just KPI projects as their performance are judged by meaningless metrics. OceanBase WAS open sourced, after the news reports and publicity, Alibaba deleted all code and replaced it with a Chinese announcement says it will no longer be an open source project.
- wanghq 5y agoSorry I didn't make myself clear. I was surprised because two databases from Alibaba are open sourced or to be open sourced.
- ioltas 5y agoLooking at their code tree, this is based on 11.2 (https://www.postgresql.org/docs/11/release-11-2.html https://www.postgresql.org/docs/11/release-11-2.html): https://github.com/alibaba/PolarDB-for-PostgreSQL/blob/master/configure.in#L20 https://github.com/alibaba/PolarDB-for-PostgreSQL/blob/maste... So this is code from two years ago missing stability fixes from upstream up to 11.12, at short sight.
- the-dude 5y agoAre there any stability fixes you have in mind? In my recollection, there is not much instability in PG releases.
- pella 5y agoIMHO: with the latest fix -> more stable search for "crash": https://www.postgresql.org/docs/release/11.3/ https://www.postgresql.org/docs/release/11.3/ ( 11x ) https://www.postgresql.org/docs/release/11.4/ https://www.postgresql.org/docs/release/11.4/ ( 4x ) https://www.postgresql.org/docs/release/11.5/ https://www.postgresql.org/docs/release/11.5/ ( 1x ) https://www.postgresql.org/docs/release/11.6/ https://www.postgresql.org/docs/release/11.6/ ( 3x ) https://www.postgresql.org/docs/release/11.7/ https://www.postgresql.org/docs/release/11.7/ ( 14x ) https://www.postgresql.org/docs/release/11.8/ https://www.postgresql.org/docs/release/11.8/ ( 5x ) https://www.postgresql.org/docs/release/11.9/ https://www.postgresql.org/docs/release/11.9/ ( 6x ) https://www.postgresql.org/docs/release/11.10/ https://www.postgresql.org/docs/release/11.10/ ( 5x ) https://www.postgresql.org/docs/release/11.11/ https://www.postgresql.org/docs/release/11.11/ ( 3x ) https://www.postgresql.org/docs/release/11.12/ https://www.postgresql.org/docs/release/11.12/ ( 5x )
- mb4nck 5y agoOTOH the PolarDB specific changes seem to be contained enough that if you decide to run it in production, you can probably just apply most of the changes from the v11 branch yourself. But I agree it's not a very good look to code-drop something on a .2 release when there's been 2,5 years of fixes.
- he0001 5y agoWhat more db’s are based on Postgres?
- jeltz 5y agoGreenplum, Postgres-XL and PipelineDB.
- ziftface 5y agoI'm still really upset that confluence bought and ruined pipeline db. It solved my problem perfectly and I haven't seen one yet that fits what I needed as well.
- ggordan 5y agoI think "based on" is more like implemented on top of, similar to timescaledb
- helltone 5y agoYellowbrick
- mkl95 5y agoTimescaleDB.
- hashhar 5y agoGreenplum Hadapt Netezza PipelineDb Postgres-XL Redshift AgensGraph TimescaleDB Fujitsu Enterprise Postgres PolarDB CrunchyBridge
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- devit 5y agoCan anybody comment on whether this actually works properly and does what it claims to do?
- killingtime74 5y agoWhy wouldn’t I use cockroachDB?
- mb4nck 5y agoI don't know, maybe you should. CockroachDB seems to be a distributed database system written in Go which has implemented a Postgres query/wire protocol compatibility layer. PolarDB is a Postgres fork actually using the Postgres codebase and extending it to a distributed database system. Maybe one day they can unfork because it's possible to implement PolarDB on top of Postgres as an extension and/or they contribute/get all their changes into Postgres core.
- rad_gruchalski 5y agoMaybe because you need the as-a-service aspect and the licensing does not allow it.
- joelp 5y agoWhat are people/companies using currently to make their postgres database distributed these days? Currently running my app + db on Heroku in the EU and would like to scale out since latency is abysmal for users in Australia and Asia.
- arthurcolle 5y agoI think Citus is one option. Or possibly TimescaleDB
- manigandham 5y agoThose are meant for a single location, not cross-region distribution.
- arthurcolle 5y agoI don't think that's right. Citus has a post from 2017 talking about allowing for horizontal scaling of a postgres instance. [0] Same thing with Timescale. [1] -------------------------------------------- [0] https://citusdata.com/blog/2017/02/16/citus61-released/ https://citusdata.com/blog/2017/02/16/citus61-released/ [1] https://blog.timescale.com/blog/timescaledb-2-0-a-multi-node-petabyte-scale-completely-free-relational-database-for-time-series/ https://blog.timescale.com/blog/timescaledb-2-0-a-multi-node...
- wirelesspotat 5y agofly.io[1] supports Postgres clusters with read replicas in different cities[2] It looks like AWS's Aurora Postgres doesn't support cross-region read replicas, but apparently their Aurora "Global Database" offering does[3] - [1] https://fly.io https://fly.io - [2] https://fly.io/docs/reference/postgres/#scaling-horizontally-adding-more-replicas https://fly.io/docs/reference/postgres/#scaling-horizontally... - [3] https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Replication.html#AuroraPostgreSQL.Replication.Replicas https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...
- 5y ago
- raarts 5y agoWhat are the differences between this and YugobyteDB?
- wejick 5y agoYugabyte is different DB technology than postgres but with PG compatibility layer, means you can use existing postgres query/application, the rest is different beast. This one is more comparable with citus, an PG extension. Meaning it runs on top postgres. However from their github page they also offer patched version of PG which I suppose offer some tighter integration with the extension.
- deleted 5y ago[deleted]
- JoelJacobson 5y agoNo commit history? It would have been interesting to follow the development work to see what changes they've made compared to PostgreSQL.
- llaolleh 5y agoThey are following the Gaussian approach. Show only the final result!
- antoncohen 5y agoThis is effectively required when open sourcing something that was previously internal. I've done it at a company. There could be company internal things that leak in old revisions of code and commit messages. Even if the latest commit on master is clean, it is really hard to know that every revisions, throughout the whole history is clean.
- JoelJacobson 5y agoGood point. In such a case, I think it would still be nice if at least trying to split the total change into a few separate commits, that builds upon each other. That way the commit log could be a good start to look at for someone who wants to understand the code base.
- JoelJacobson 5y agoWith the first commit being the specific commit from the PostgreSQL repo which they base their work on, i.e. that they forked from.
- rad_gruchalski 5y agoI'd love to try this out. What I'm missing though is at least some Docker based deployment not involving building stuff from sources and reinventing distributed architecture myself.