levkk

8 ms·

91.
by levkk 7mo ago
All queries run inside transactions, and a slow lane like S3 will cause delays, which will in turn block vacuum and cause more problems than it will solve. Most deployments of Postgres (e.g., RDS) won't let you install custom extension
92.
by levkk 7mo ago
See prepared statements.
93.
by levkk 7mo ago
Models don't learn. They retrain them periodically, but junior engineers learn much faster and constantly improve. If you stop learning, you will only be as good as the model. I've been coding (software engineering, I guess) for c
94.
by levkk 7mo ago
Yeah! Take a look at our Docker demo in GitHub: https://github.com/pgdogdev/pgdog
95.
by levkk 7mo ago
You can I believe. We only support BIGINT, VARCHAR and UUID for sharding, but all other data types are completely fine for passthrough, i.e. to be included and used in your queries.
96.
by levkk 7mo ago
General statement about adoption. Last time we made a Show HN (9 months ago), it was a POC, running on my local. Now we're used in production by some pretty big companies, which is exciting!
97.
by levkk 7mo ago
> Are there some specific standard Postgres test suites you run PgDog through to ensure it's compliant with Postgres standards? That's right. We have many levels of testing: unit, integration, and acceptance, where we run the s
98.
by levkk 7mo ago
That's exactly right, it's both of those. More containers / services means more connections to the DB, which themselves need to be pooled. More requests to the app require more connections as well.
99.
by levkk 7mo ago
Technically yes. We only support BIGINT (and all other integers), VARCHAR and UUID for sharding keys, but we'll happily pass through any other data. If we need to process it, we'll need to parse it. To be clear: you can include Po
100.
by levkk 7mo ago
A couple options come to mind: 1. Replicate shards into one beefy database and use that. Replication is cheaper than individual statements, so this can work for a while. The sink can be Postgres or another database like Clickhouse. At Insta
101.
by levkk 7mo ago
I would say, over 100 Postgres connections, consider getting a connection pooler. Requests per second is highly variable. Postgres can serve a lot of them, as long as you keep the number of server connections low - that's what the pool
102.
by levkk 7mo ago
The current behavior unfortunately is to just let it through and return an incorrect result. We are adding more checks here and rely heavily on early adopters to have a decent test suite before launching their apps to prod. That being said,
103.
by levkk 7mo ago
So many, where to begin. 1. People don't design schemas to be sharded, although many gravitate towards a common key, e.g. user_id or country_id or tenant_it or customer_id. Once that happens, sharding becomes easier. 2. Postgres provid
104.
by levkk 7mo ago
PostgREST is a translation layer: you use HTTP methods, inputs and outputs, to interact with Postgres, the database. It's a replacement for SQL, the language, which happens to also have a load balancer. Their load balancer is still at
105.
by levkk 7mo ago
1. Yup, we support online resharding, so you don't need to deploy this until you have to. 2. That's right, we broadcast the DDL to all shards in the configuration. If two-phase commit [1] is enabled, you have a strong guarantee th
106.
by levkk 8mo ago
Absolutely. We do weekly releases here: https://github.com/pgdogdev/pgdog/releases . Each release includes a detailed changelog. I believe you can use an RSS reader if those are still in vogue, e.g.: https:/&
107.
by levkk 8mo ago
I think we may have fixed this 3 weeks ago: https://github.com/pgdogdev/pgdog/pull/744 Might be worth another try. If not, a GitHub issue with more specifics would be great, and we'll take a look. Also,
108.
by levkk 8mo ago
I'll need a bit more info about your use case to answer. We use logical replication to move data between shards, with the intention of creating new shards. This is managed by PgDog. We are building a lot of tooling here, and a lot of i
109.
by levkk 8mo ago
Not currently, but we can add this. One thing we have to be careful of is to not retry requests that are executing inside transactions, but otherwise this would be a great feature.
110.
by levkk 8mo ago
It can be silent, but usually it's loud and confusing because people do something like this (Rails example): user = User.create(email: "test@test.com") SendWelcomeEmail.perform_later(user.id) And the job code fet
111.
by levkk 8mo ago
It shards it as well. We handle schema sync, moving table data (in parallel), setting up logical replication, and application traffic cutover. The zero-downtime resharding is currently WIP, working on the PR as we speak: https://
112.
by levkk 8mo ago
Not really, replication lag is generally an accepted trade-off. Sync replication is rarely worth it, since you take a 30% performance hit on commits and add more single points of failure. We will add some replication lag-based routing soon.
113.
by levkk 8mo ago
We have all kinds, it's not specific to any particular sector. That's kind of the beauty for building for Postgres - everyone uses it in some capacity! My general advice is, once you see more than 100 connections on your database,
114.
by levkk 8mo ago
Subms typically, yeah. We measured the average latency between nodes in the same AZ (e.g., AWS availability zone) to be less than one ms, so you need to account for one extra hop and processing time by PgDog, which is typically fast. That b
115.
by levkk 8mo ago
It feels better now, but we still need to add crash protection - in case PgDog itself crashes, we need to restore in-progress 2pc transaction records from a durable medium. We will add this very soon.
116.
Show HN: PgDog – Scale Postgres without changing the app(github.com)
326 points by levkk 8mo ago|63 comments
117.
by levkk 8mo ago
I'm curious about your reasoning. Jira/Trello etc. are like $10/mo/seat, why bother rewriting them from scratch? You'll spend more in tokens doing so. Same for gmail/google calendar, what's the ROI? Those
118.
PgDog: Connection pooler, load balancer and sharder for PostgreSQL
1 points by levkk 8mo ago|0 comments
119.
by levkk 8mo ago
I think the right way to handle this as a repository owner is to close the PR and block the "contributor". Engaging with an AI bot in conversation is pointless: it's not sentient, it just takes tokens in, prints tokens out, a
120.
by levkk 8mo ago
One of the many problems PgDog will solve for you!

More ›