2 ms·

Is it possible not to use a PostgreSQL database? For example would it run with SQLite? The goal is to improve developer experience.

qianli_cs 2y ago
Co-founder here! No current plans to support SQLite. We picked Postgres because of its huge ecosystem--you can use DBOS with any PostgreSQL-compatible database (Supabase, Neon, Aurora, Cockroach...) and with any Postgres extension (here's an example app using pgvector: https://github.com/dbos-inc/dbos-demo-apps/tree/main/python/llamabot https://github.com/dbos-inc/dbos-demo-apps/tree/main/python/...)
- threecheese 2y ago
  (After a quick look at the code) Is this due to concurrency (writes)? It looks like this architecture supports multiple executors, and I would imagine you require transactional guards to ensure consistency. I really like this interface btw, the complexity is hidden very well and from reading your docs it remains accessible if you need to dig deeper than a decorator. And how the heck are you maintaining Typescript and Python copies? lol
  - qianli_cs 2y ago
    Thanks for your kind words! We're focusing on Postgres because an important scenario for durable execution is serverless computing, which won't work with an embedded database.
    - threecheese 2y ago
      I am sure you are aware of this, but if not: there are some emerging technologies around embedded database scale-out using CRDT and other replication protocols that would support various “serverless” (as in decentralized) topologies. PGLite, sqlite-cr, libSQL et al. I am informally looking at for serverless executor agents that do not need to coalesce around a central database instance (“server-full”). I am sure you tested something like this, I would guess that classic/CDC replication lag would throw a big wrench into an attempt to orchestrate disconnected remote executors, I am hoping that in a peer to peer topology this new tech will have low enough sync latencies to be useful. Best of luck with DBOS! You have an amazing team.
sitkack 2y ago
You can now run Wasm builds of PostgreSQL that will get you everything you like about SQLite. https://github.com/electric-sql/pglite https://github.com/electric-sql/pglite

qianli_cs 2y ago

Co-founder here! No current plans to support SQLite. We picked Postgres because of its huge ecosystem--you can use DBOS with any PostgreSQL-compatible database (Supabase, Neon, Aurora, Cockroach...) and with any Postgres extension (here's an example app using pgvector: https://github.com/dbos-inc/dbos-demo-apps/tree/main/python/llamabot https://github.com/dbos-inc/dbos-demo-apps/tree/main/python/...)

threecheese 2y ago
(After a quick look at the code) Is this due to concurrency (writes)? It looks like this architecture supports multiple executors, and I would imagine you require transactional guards to ensure consistency. I really like this interface btw, the complexity is hidden very well and from reading your docs it remains accessible if you need to dig deeper than a decorator. And how the heck are you maintaining Typescript and Python copies? lol
- qianli_cs 2y ago
  Thanks for your kind words! We're focusing on Postgres because an important scenario for durable execution is serverless computing, which won't work with an embedded database.
  - threecheese 2y ago
    I am sure you are aware of this, but if not: there are some emerging technologies around embedded database scale-out using CRDT and other replication protocols that would support various “serverless” (as in decentralized) topologies. PGLite, sqlite-cr, libSQL et al. I am informally looking at for serverless executor agents that do not need to coalesce around a central database instance (“server-full”). I am sure you tested something like this, I would guess that classic/CDC replication lag would throw a big wrench into an attempt to orchestrate disconnected remote executors, I am hoping that in a peer to peer topology this new tech will have low enough sync latencies to be useful. Best of luck with DBOS! You have an amazing team.

sitkack 2y ago

You can now run Wasm builds of PostgreSQL that will get you everything you like about SQLite. https://github.com/electric-sql/pglite https://github.com/electric-sql/pglite