KraftyOne

3 ms·

1.
Taking Out the Trash in Postgres(dbos.dev)
1 points by KraftyOne 2mo ago|0 comments
2.
Postgres SELECT DISTINCT does not scale(dbos.dev)
3 points by KraftyOne 2mo ago|0 comments
3.
Postgres SELECT DISTINCT does not scale(dbos.dev)
3 points by KraftyOne 3mo ago|0 comments
4.
Postgres SELECT DISTINCT does not scale(dbos.dev)
4 points by KraftyOne 3mo ago|0 comments
5.
Anyscale Joins NScale(anyscale.com)
3 points by KraftyOne 3mo ago|0 comments
6.
by KraftyOne 3mo ago
This is something we mention in the third section of the article: dead tuples and autovacuum caused real performance hits, which we (partially) mitigated through index optimization (minimizing the number and size of indexes, and making inde
7.
Making Postgres queues scale(dbos.dev)
128 points by KraftyOne 3mo ago|34 comments
8.
by KraftyOne 4mo ago
The core optimization is to buffer notifications in-memory and send them in a batch instead of sending them as part of every transaction. So that's a general-purpose optimization for Postgres apps using LISTEN/NOTIFY.
9.
by KraftyOne 4mo ago
If you mean the optimizations coming in Postgres 19, the original post addresses this: > As an aside, there’s been some online discussion of a Postgres patch ( https://github.com/postgres/postgres/commit/282
10.
by KraftyOne 4mo ago
To be clear, it's not a custom patch to pg itself, but an application-side buffering and batching optimization.
11.
Postgres LISTEN/NOTIFY actually scales(dbos.dev)
373 points by KraftyOne 4mo ago|84 comments
12.
Just Use Postgres for Job Queues(dbos.dev)
7 points by KraftyOne 4mo ago|0 comments
13.
by KraftyOne 4mo ago
Outbox's power is that it turns an atomicity problem into an idempotency problem. You atomically write to the outbox, then you have an idempotent "workflow" that processes events from the outbox. This turns "at most once
14.
by KraftyOne 4mo ago
You still need idempotency for side effects outside your database, that's true (and fundamental to durability). But now you get exactly-once semantics for operations on your database, which can be quite valuable if your workflow perfor
15.
by KraftyOne 4mo ago
That's what the post is about! Once you're doing that, you really do have transactions between the state and the queue.
16.
by KraftyOne 4mo ago
Every item will be written to the queue exactly once (as the update is transactional). Queue processing may need at-least-once semantics, yes, depending on what exactly you're doing.
17.
by KraftyOne 4mo ago
You build a distributed system on top of this! For example, you may have many distributed workers durably executing workflows from the Postgres-backed task queue. The Postgres transactions allow you to atomically perform operations spanning
18.
by KraftyOne 4mo ago
Yes, the core design is building a workflow system on a database--essentially, replacing the central orchestrator most workflow systems use with a Postgres database. This previous blog post goes into more detail: https://www.dbos
19.
by KraftyOne 4mo ago
The key is that the UDF's enqueue is transactional with the database update. Let's say the database update is inserting a new order. This provides the guarantee that if a new order is inserted, a job to process the order is also e
20.
by KraftyOne 4mo ago
Exactly! It's a distributed system, with many processes performing work in parallel, with a central database as a coordination point, used as little as possible. A mutex wouldn't get quite the same performance :)
21.
Postgres transactions are a distributed systems superpower(dbos.dev)
232 points by KraftyOne 4mo ago|96 comments
22.
by KraftyOne 5mo ago
Unfortunately, DSQL is still lacking many core Postgres features (most notably foreign keys) that DBOS and DBOSify depend on. DBOS works with most flavors of Postgres though, including regular Aurora/RDS, Cloud SQL, AlloyDB, Supabase,
23.
by KraftyOne 5mo ago
Take any Temporal example and swap the imports and connection strings, it's a drop-in replacement :)
24.
by KraftyOne 5mo ago
We'd need DBOS in C# first! It's something we're considering for the future--if you're working in C#, happy to chat about it, reach out at peter.kraft@dbos.dev.
25.
by KraftyOne 5mo ago
Exactly, Postgres as an orchestrator instead of a Temporal server as an orchestrator. This page goes into more detail (for DBOS, but DBOSify is the same principle but using the Temporal API directly): https://docs.dbos.dev/e
26.
by KraftyOne 5mo ago
I'm very sorry about this. I'll figure out why the form wasn't responded to. In the meantime, email me directly at peter.kraft@dbos.dev.
27.
by KraftyOne 5mo ago
Yes it can (with documented exceptions)! The suite is here: https://github.com/dbos-inc/dbosify-py/tree/main/tests/confo... Here's the documentation: https://github.com/dbos-in
28.
Just Use Postgres for Task Queues(dbos.dev)
2 points by KraftyOne 5mo ago|0 comments
29.
Is this blog written by AI?(brooker.co.za)
5 points by KraftyOne 5mo ago|0 comments
30.
by KraftyOne 5mo ago
Hi HN, I'm Peter, creator of DBOSify. Here to answer any questions you have!

More ›