3 ms·

Looks really nice, but I'm really curious about what hurdles you encountered in making a native WASM version of Postgres. It seems that both the performance and

Looks really nice, but I'm really curious about what hurdles you encountered in making a native WASM version of Postgres. It seems that both the performance and binary size suffer immensely from the VM running underneath. For example, if I compare to DuckDB-WASM [1] there is an immense difference in load time and query execution speed. Are there future plans at creating a native WASM version of Postgres? Making it run many times faster would certainly open up a lot more use cases. (Disclaimer: I work on DuckDB, but have not worked on the WASM version myself) [1] https://shell.duckdb.org https://shell.duckdb.org

kiwicopple 4y ago
congrats to the DuckDB team for their WASM version. I've been following it from afar and it's very impressive. > what hurdles you encountered in making a native WASM I'm sure Mark & Peter can jump in with specifics but mostly it was due to complexity - there it probably can be done it's just that we took the path of least resistance. > Are there future plans at creating a native WASM version of Postgres We'd like that. If anyone would like to collaborate with Supabase + Snaplet to create a more "native WASM" version then please reach out
- burggraf 4y ago
  Supabase developer here. Yes we'd love to collaborate on a native WASM version. I tried this first before starting on this project and I didn't get very far. I'll have to go back to my tests on that, but it seemed like WASM wasn't yet capable of compiling Postgres due do some libraries not being supported. Hopefully we can find workarounds or push WASM into a new era of compatibility :)
pistoriusp 4y ago
Snaplet person here: Having a native WASM version would be a huge win for everyone, especially is storage is decoupled from compute! At the moment the CPU and memory snapshot of the VM (with Postgres) is 12 MB, and subsequent reloads are cached. So yeah, not the worst, but not great. An optimization is that we're using 9P filesystem. So accessing anything on disk is lazily loaded over the network. > Are there future plans at creating a native WASM version of Postgres? Yup! I think that should be the goal, and we (Supabase & Snaplet) would be very happy to work with anyone that wants to build towards that.
- chrisjc 4y ago
  > especially is storage is decoupled from compute! This would be amazing! I can imagine a situation where external tables are managed by some MPP, and a WASM compute engine (Postgres, DuckDB, etc) would be able to at least read subsets/partitions of the full external table. I wonder if the work required to make a native WASM Postgres would have to be split up into efforts for row-based vs column-based. Selfishly, I would love to have access to a column-based version first.
  - pistoriusp 4y ago
    We actually kinda see parts of this in https://github.com/superfly/litefs https://github.com/superfly/litefs, albeit for SQLite. And here we see some ideas forming around "pluggable storage for PostgresQL": https://wiki.postgresql.org/wiki/Future_of_storage#Pluggable_storage_for_tables https://wiki.postgresql.org/wiki/Future_of_storage#Pluggable... Seriously! If any of this sounds interesting to build, reach out, and we'll make it happen!

kiwicopple 4y ago

congrats to the DuckDB team for their WASM version. I've been following it from afar and it's very impressive. > what hurdles you encountered in making a native WASM I'm sure Mark & Peter can jump in with specifics but mostly it was due to complexity - there it probably can be done it's just that we took the path of least resistance. > Are there future plans at creating a native WASM version of Postgres We'd like that. If anyone would like to collaborate with Supabase + Snaplet to create a more "native WASM" version then please reach out

burggraf 4y ago
Supabase developer here. Yes we'd love to collaborate on a native WASM version. I tried this first before starting on this project and I didn't get very far. I'll have to go back to my tests on that, but it seemed like WASM wasn't yet capable of compiling Postgres due do some libraries not being supported. Hopefully we can find workarounds or push WASM into a new era of compatibility :)

pistoriusp 4y ago

Snaplet person here: Having a native WASM version would be a huge win for everyone, especially is storage is decoupled from compute! At the moment the CPU and memory snapshot of the VM (with Postgres) is 12 MB, and subsequent reloads are cached. So yeah, not the worst, but not great. An optimization is that we're using 9P filesystem. So accessing anything on disk is lazily loaded over the network. > Are there future plans at creating a native WASM version of Postgres? Yup! I think that should be the goal, and we (Supabase & Snaplet) would be very happy to work with anyone that wants to build towards that.

chrisjc 4y ago
> especially is storage is decoupled from compute! This would be amazing! I can imagine a situation where external tables are managed by some MPP, and a WASM compute engine (Postgres, DuckDB, etc) would be able to at least read subsets/partitions of the full external table. I wonder if the work required to make a native WASM Postgres would have to be split up into efforts for row-based vs column-based. Selfishly, I would love to have access to a column-based version first.
- pistoriusp 4y ago
  We actually kinda see parts of this in https://github.com/superfly/litefs https://github.com/superfly/litefs, albeit for SQLite. And here we see some ideas forming around "pluggable storage for PostgresQL": https://wiki.postgresql.org/wiki/Future_of_storage#Pluggable_storage_for_tables https://wiki.postgresql.org/wiki/Future_of_storage#Pluggable... Seriously! If any of this sounds interesting to build, reach out, and we'll make it happen!