8 ms·
SPQR 1.3.0: a production-ready system for horizontal scaling of PostgreSQL
- soleri17 3y agoDo you have some plans for k8s operator?
- denchick 3y agohttps://github.com/pg-sharding/spqr/issues/571 https://github.com/pg-sharding/spqr/issues/571
- u320 3y agoDoes it handle byzantine failures?
- gclawes 3y agoNo, only Roman failures
- hot_gril 3y agoSo during a Parthian event, they lose the Crassus.
- bheadmaster 3y agoLike the one where they failed to conquer that village of Gauls?
- SketchySeaBeast 3y agoAs usual, Asterixes are to be avoided.
- alexvoda 3y agoBesides, there are plenty of Obelixs to be found in Egypt. There is no need to get your phalic stone monuments from the Gauls. (I expected Roman puns and was not disappointed)
- lenerdenator 3y agoSounds like a pretty germane feature to have.
- skrebbel 3y agoParent means Eastern Roman failures
- m0rshu 3y ago[dead]
- deleted 3y ago[deleted]
- geenat 3y agoWhy use SPQR over Citus? Vitess?
- feverzsj 3y agoSPQR, as the name suggests, is just a query router.
- calvinmorrison 3y agoYes, that's what the name suggests
- verandaguy 3y agoThe name also suggests plans to go to war with the Carthaginians.
- jitl 3y agoVitess is a MySQL cluster manager, it doesn’t support Postgres. https://github.com/vitessio/vitess/issues/7084 https://github.com/vitessio/vitess/issues/7084
- lebski88 3y agoThe fact that it's a router and not a PG extension is helpful if you are using cloud hosted postgres and don't want to get into the business of managing backups etc yourselves. Only azure supports Citus as far as I know.
- 3y ago
- bookofjoe 3y agoSPQR https://en.wikipedia.org/wiki/SPQR https://en.wikipedia.org/wiki/SPQR
- deleted 3y ago[deleted]
- Mortiffer 3y agoA comparison to https://www.citusdata.com/ https://www.citusdata.com/ would be nice
- mdaniel 3y agofor one thing: https://github.com/pg-sharding/spqr/blob/1.3.0/LICENSE https://github.com/pg-sharding/spqr/blob/1.3.0/LICENSE (BSD2) https://github.com/citusdata/citus/blob/v12.1.2/LICENSE https://github.com/citusdata/citus/blob/v12.1.2/LICENSE (AGPLv3)
- waldrews 3y agoI still have a hard time understanding where the AGPL copyleft feature kicks in on a product like Citus. Are you safe just deploying it as part of your backend? Like, do you have to actually touch Citus's own code to trigger it? What if you start using some sprocs or sample connectivity code? Anybody know a good set of guidelines?
- wavemode 3y agoThis is the entirety of the text which was added to the AGPL causing it to differ from the plain GPL: > Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software. This Corresponding Source shall include the Corresponding Source for any work covered by version 3 of the GNU General Public License that is incorporated pursuant to the following paragraph. In other words, if you deploy a modified version of Citus, your modified version is AGPL licensed, and thus you must provide the source code of this modified version to all users who interact with it remotely (e.g. through your web application). What it does not state is that you must provide the source code of your entire web application just because you deployed a modified version of Citus, nor that your web application becomes an AGPL-licensed derived work of Citus because you used it over a network. The AGPL also does not require anything at all from you other than the plain GPL's basic requirements if the version of Citus you deploy is unmodified. (These are all incredibly common misconceptions on the internet, by people who've never read the license nor the GNU website.)
- vb-8448 3y agosono pazzi questi romani
- vincnetas 3y ago"On the first page where we find the map of France we see a Roman banner with S.P.Q.R. which means "Senatus Populus Que Romanus" (The senate and people of Rome). The Italian translator made a great pun about it and translated it into "Sono Pazzi questi Romani". The latter can be translated to "These Romans are crazy"." https://www.asterix-obelix.nl/index.php?page=hjh/dos-italy.inc https://www.asterix-obelix.nl/index.php?page=hjh/dos-italy.i...
- wlll 3y agoUnrelated, but you reminded me of this clip from The Life of Brian: https://www.youtube.com/watch?v=wjOfQfxmTLQ https://www.youtube.com/watch?v=wjOfQfxmTLQ It's funnier if you know that the ear-twisting Roman at the start was basically a caricature of many Latin teachers that were around when Latin was taught more widely in schools (such as when the Monty Python team were young).
- davemo 3y agoVeni, Vidi, Vici
- deleted 3y ago[deleted]
- jimmywetnips 3y agoNice
- ThinkBeat 3y agoHow far can one get these days with vertical scaling of Postgres? I dont know how well the engine could make use of the extra resources. If it scales well vertically, I expect it to cover 80% - 95% of use cases for people using Postgres. Probably with less complexity, and less overhead than scaling horziontally
- endymi0n 3y agoFar. As in, really, really far. We started out with Postgres because it was just the simple and sensible option for a production prototype, and when somebody came around telling me we need something more scalable recently, I calculated that there's not even enough addressable market in the world for our business for more than 4x our size. That's exactly the two remaining vertical doublings of our DB instance (to ridiculously looking RAM & CPU numbers) we could still do in case we need it. Other than that, thanks to a lot of the recent work on connection handling and concurrency since PG 11, Postgres is getting better and better actually using these additional resources well: https://www.enterprisedb.com/blog/performance-comparison-major-PostgreSQL-versions https://www.enterprisedb.com/blog/performance-comparison-maj...
- ThinkBeat 3y agoSeveral years ago I was a contractor on a project for that would sip up data from all forms of sensor and moving objects around a large city. It was decided that we needed Kafka and a few similar tools to handle it. It was not hard to calculate the current max traffic or estimate the traffic growth over the next 10 years. I did a demo of the system running on my laptop (all of it) + Postgres handling 100x the current data without too much difficulty. Still they went with the "scale" solution because it was the right design. (and of course the consultants and me got quite lot more work todo so made a good deal more money)
- mdasen 3y agoAt this point you can get 24TB of RAM in an EC2 instance (along with 448 vCPUs, 100Gbps of network bandwidth and 38Gbps of EBS bandwidth). That won't scale forever, but Stack Overflow has been running on a single primary/standby setup with 1.5TB of RAM so that would be 16x Stack Overflow's RAM. I think a lot of work goes into horizontal scaling which is necessary at a certain scale, but very few people actually get anywhere near that scale. It can be important to understand which things are needed at your scale and where you can simply buy some beefier hardware. I've been at places where people run a dozen sharded DB servers with each server having 16GB of RAM. Maybe that's resume-driven-development where someone wants to say they've done that.
- lenage 3y agoHappy to see so many tools created for PostgreSQL, Thanks for your great work
- AtlasBarfed 3y agoSQL is problematic to horizontally scale, joins in particular. I see nothing about partition tolerance, so I assume it isn't at all. Give me Aphyr tests or there is no reason to pay attention.