levkk

15 ms·

211.
by levkk 1y ago
You're right. Good catch, I'll fix this in the article.
212.
by levkk 1y ago
We're using it to rewrite queries too. It's a pretty cool library.
213.
Hacking the Postgres wire protocol(pgdog.dev)
229 points by levkk 1y ago|50 comments
214.
Vertical Sharding Sucks(pgdog.dev)
2 points by levkk 1y ago|0 comments
215.
by levkk 2y ago
Not usually, that's considered an potential attack vector I believe. You're looking to minimize information leakage.
216.
by levkk 2y ago
Pretty sure kernel doesn't have to fsync on close. In fact, you don't want it to, otherwise you're limiting the performance of your page cache. So fsync on install for dpkg makes perfect sense.
217.
by levkk 2y ago
Good call-out. The article mentions using a representative sample, but I should definitely put it somewhere on top.
218.
Sharding Pgvector(pgdog.dev)
86 points by levkk 2y ago|10 comments
219.
by levkk 2y ago
I ended up adding support for GROUP BY: https://github.com/pgdogdev/pgdog/pull/43 I had it in the back of my mind for a while, nice to have it in code. Works pretty well, as long as columns in GROUP BY are pr
220.
by levkk 2y ago
Replace it with a function that makes sure the id coming out matches the sharding schema. Assuming it's coming from a sequence, we're consuming it until we get a matching number. It would be good to know what is behind the generat
221.
by levkk 2y ago
The goal for this project is to be analogous to Vitess for Postgres.
222.
by levkk 2y ago
No worries. It's early days, the code and docs are just a few months in the making. I'm happy to keep you updated on the progress. If you want, send your contact info to hi@pgdog.dev. - Lev
223.
by levkk 2y ago
Aggregates are a work in progress. We're going to implement them in this order: 1. count 2. max, min, sum 3. avg (needs a query rewrite to include count) Eventually, we'll do all of these: https://www.postgresql.org&#x2
224.
by levkk 2y ago
I was thinking of the Instagram post years ago. And maybe the Instacart one.
225.
by levkk 2y ago
We actually found Aurora to be about 3x slower than community PG for small queries. That was back then, maybe things are better now. Migrating to another database (and Aurora is Postgres-compatible, it's not Postgres) is very risky whe
226.
by levkk 2y ago
We use the Postgres parser directly, thanks to the great work of pg_query [1]. [1] https://github.com/pganalyze/pg_query.rs
227.
by levkk 2y ago
I'm glad you brought up the migration, because one of the main goals behind our project is to automate migrating to a sharded deployment. You can think of your DB as the base case (num_shards = 1), and PgDog as the recursive solution.
228.
by levkk 2y ago
We certainly have a way to go to support all cross-shard use cases, especially complex aggregates (like percentiles). In OLTP, where PgDog will focus on first, it's good to have a sharding key and a single shard in mind, 99% of the ti
229.
by levkk 2y ago
Great question and we will publish something for this later on. TLDR: Citus doesn't provide an easy migration path for existing DBs and the managed deployments are pretty much exclusive to Azure. A large percentage of production DBs ru
230.
by levkk 2y ago
It's not. We tried. Plus, it doesn't work on RDS, where most of production databases are. I think Citus was a great first step in the right direction, but it's time to scale the 99% of databases that don't run on Azure C
231.
Making Postgres scale(pgdog.dev)
225 points by levkk 2y ago|188 comments
232.
by levkk 2y ago
We care, we just can't spend the resources self-hosting business critical infrastructure. It'll break, get AI-DoSed, or will have an expired cert. It's a boring and risky problem to solve. Cloudflare provides a really good se
233.
by levkk 2y ago
For a programming language that makes unused variables a compiler error, I'm surprised there is no clippy/rubocop/black/cargo fmt/clang fmt built-in.
234.
by levkk 2y ago
What remains to be seen if your venture (and others like it) will translate into hundreds of billions of new economy. That was the effect of the iPhone, cloud, and Google Search (analogies used in the article). That's the difference be
235.
by levkk 2y ago
If my Android (or IPhone) disappeared tomorrow, I would feel like I time traveled back a century. If Google search was gone, I wouldn't be able to do my job anymore. If the cloud disappeared, I wouldn't be able to build apps anymo
236.
Working with the Postgres Protocol(levkk.bearblog.dev)
3 points by levkk 2y ago|0 comments
237.
Sharding Postgres with Logical Replication(levkk.bearblog.dev)
2 points by levkk 2y ago|0 comments
238.
by levkk 2y ago
TIL. Thanks!
239.
by levkk 2y ago
You just need to quote your identifiers with double quotes, e.g.: SELECT * FROM "MyTable" You can even use reserved SQL keywords in table/column names as long as they are double-quoted.
240.
Show HN: PgDog, Load Balancer for PostgreSQL(github.com)
2 points by levkk 2y ago|1 comments

More ›