4 ms·
I'd seen this running in a browser before (the ~3MB download is really impressive for that), but I hadn't clocked that it runs server-side with Node.js and Bun
by simonw 2y ago
I'd seen this running in a browser before (the ~3MB download is really impressive for that), but I hadn't clocked that it runs server-side with Node.js and Bun as well: https://pglite.dev/docs/ https://pglite.dev/docs/
Since that's still not spinning up an actual network server, that feels like it's an alternative to SQLite - you can spin up a full in-process PostgreSQL implementation, that persists to disk, as part of an existing Node.js/Bun application.
That's really interesting!
I'd love to use this from Python, via something like https://github.com/wasmerio/wasmer-python https://github.com/wasmerio/wasmer-python or https://github.com/bytecodealliance/wasmtime-py https://github.com/bytecodealliance/wasmtime-py - has anyone run PGlite via one of those wrappers yet?
- samwillis 2y agoGetting PGlite working with other languages is very high on my list. We are working on two approaches: - a WASI build with a lower level api that users can wrap with a higher level api. - a "libpglite" that can be linked to by any native language. The WASI build is likely a WASM build of this. Most languages already have a Postgres wire protocol implementation and so wrapping a low level API that reads/emits this is relatively easy - it's what the JS side of PGlite does.
- sitkack 2y agoI'd love to see this for SQLite and DuckDB at the same time. I could see them using a common wasi filesystem shim so they can transparently do range requests over http blobs. Do you have a branch for this work?
- samwillis 2y agoNo branch yet, we're still in the early research and experimental stage.
- rubenfiszel 2y agoIf you can make it work with Rust, and compatible with wasm targets as well, it opens a huge field of possibilities. Windmill.dev is a workflow engine based fully on postgresql. This would be the missing piece to offer a local development workflow that doesn't require spinning a full pg.
- tracker1 2y agoDefinitely interesting... maybe someone more familiar with the licenses involved can chime in on how this might impact a project/solution and compare/contrast to something like embedded FirebirdSQL. For example, can you use this in/with a commercial project without releasing source? If you embed this, will you need to change your license to match pg/fb? aside: Mostly asking out of curiosity if anyone knows already, if I actually had an immediate need, would be reviewing this of my own accord. I often dislike a lot of the simplicity behind SQLite in very similar ways as to things I don't care for with MySQL/MariaDB.
- abrichr 2y agoAs a workaround, maybe something like this could work: db.js: const { PGlite } = require('@electric-sql/pglite'); const db = new PGlite(); async function execSQL(sql) { await db.exec(sql); return { success: true }; } async function querySQL(sql, params) { const ret = await db.query(sql, params); return ret.rows; } // Command-line interface logic const action = process.argv[2]; const sql = process.argv[3]; const params = process.argv.slice(4); if (action === 'exec') { execSQL(sql).then(result => { console.log(JSON.stringify(result)); process.exit(); }); } else if (action === 'query') { querySQL(sql, params).then(result => { console.log(JSON.stringify(result)); process.exit(); }); } main.py: import subprocess import json def execute_sql(sql: str) -> dict: result = subprocess.run( ['node', 'db.js', 'exec', sql], stdout=subprocess.PIPE, text=True ) return json.loads(result.stdout) def query_sql(sql: str, params: list) -> list: result = subprocess.run( ['node', 'db.js', 'query', sql] + params, stdout=subprocess.PIPE, text=True ) return json.loads(result.stdout) # Example Usage create_table_sql = """ CREATE TABLE IF NOT EXISTS todo ( id SERIAL PRIMARY KEY, task TEXT, done BOOLEAN DEFAULT false ); """ execute_sql(create_table_sql) query_result = query_sql("SELECT * FROM todo WHERE id = $1", ["1"]) print(query_result)
- samwillis 2y agoWe have the (accidentally undocumented) execProtocolRaw api that lets you execute Postgres wire protocol messages. If you use that you can then use a Python Postgres wire protocol lib to get all the type support.
- deleted 2y ago[deleted]
- networked 2y agoI'd also love to try PGlite in Python. While reading this thread, I realized that you could already access PGlite in Pyodide. Pyodide is a Wasm port of CPython. It can work with JavaScript objects through proxies. Here is a demo. I have run the code in current Node.js 18, Deno 1, and Bun 1. import { PGlite } from "@electric-sql/pglite"; import { loadPyodide } from "pyodide"; globalThis.db = new PGlite(); let pyodide = await loadPyodide(); console.log( await pyodide.runPythonAsync(` import js ret = await js.db.query(""" SELECT 'Hello, world!' AS "greeting"; """) ret.rows[0].greeting `), ); It works on my machine: > npm init -y > npm install '@electric-sql/pglite@==0.2.0' > npm install 'pyodide@==0.26.2' > node index.mjs prerun(C-node) worker= false Running in main thread, faking onCustomMessage Hello, world!
- WuxiFingerHold 2y agoIt'll be interesting how well this works out in practice. I don't know how they modified the persistence layer of PG, but I could imagine that there are limitations compared to running PG in a docker container. SQLite doesn't have those limitations.