4 ms·

Really excited about the future of DuckDB: 1. DuckLake is the best datalake spec and their team is improving on the extension rapidly. 2. With DuckDB WASM, yo

Really excited about the future of DuckDB: 1. DuckLake is the best datalake spec and their team is improving on the extension rapidly. 2. With DuckDB WASM, you can make apps that would normally have 2 to 3 second latency for network calls work in < 200ms. We use it as our built-in datalake at Definite and couldn't be happier with it. 0 - https://ducklake.select/ https://ducklake.select/ 1 - https://www.definite.app/blog/ducklake https://www.definite.app/blog/ducklake

curiousgal 1y ago
I am curious is anyone using DuckDB in prod?
- mgaunard 1y ago
  of course, why wouldn't you? I was using it even before it hit 1.0
- mritchie712 1y ago
  Yes, we run DuckDB + DuckLake in prod for https://www.definite.app/ https://www.definite.app/
  - fluxkernel 1y ago
    Is DuckLake compatible with Iceberg? I remember they create a new catalog.
    - mlnj 1y ago
      Yes, you can use INSTALL iceberg; to start working with iceberg https://duckdb.org/docs/stable/core_extensions/iceberg/overview https://duckdb.org/docs/stable/core_extensions/iceberg/overv...
- matsonj 1y ago
  Sure - https://motherduck.com/blog/15-companies-duckdb-in-prod/ https://motherduck.com/blog/15-companies-duckdb-in-prod/
_ea1k 1y ago
Agreed. I just wish that the vector support were fully supported. It has been experimental for a long time.
geysersam 1y ago
I love duckdb but I don't understand your second point. How does duckdb wasm decrease the latency of network calls in your app?
- benrutter 1y ago
  Not OP, but depending on your use case, client side operations could be a big deal. Something like a dashboard app for instance, might have callbacks where it needs to run a new SQL query based on input, but be small enough to make loading all data onto menory and querying locally a mich faster option. Obviously ymmv and if you have a dashboard with a heavy data set under the hood, your gonna make a lot of users unhappy fast by doing that.
- mritchie712 1y ago
  for many uses cases, you can cache a dataset client side that can be re-queried to avoid network calls
adsharma 1y ago
Love the DuckLake spec. Now there is something similar for Graphs. https://adsharma.github.io/graph-archiving/ https://adsharma.github.io/graph-archiving/ https://adsharma.github.io/beating-the-CAP-theorem-for-graphs/ https://adsharma.github.io/beating-the-CAP-theorem-for-graph... The catalog is based on the now archived kuzu graph db project. Development continuing here: https://github.com/LadybugDB/ladybug https://github.com/LadybugDB/ladybug

curiousgal 1y ago

I am curious is anyone using DuckDB in prod?

mgaunard 1y ago
of course, why wouldn't you? I was using it even before it hit 1.0
mritchie712 1y ago
Yes, we run DuckDB + DuckLake in prod for https://www.definite.app/ https://www.definite.app/
- fluxkernel 1y ago
  Is DuckLake compatible with Iceberg? I remember they create a new catalog.
  - mlnj 1y ago
    Yes, you can use INSTALL iceberg; to start working with iceberg https://duckdb.org/docs/stable/core_extensions/iceberg/overview https://duckdb.org/docs/stable/core_extensions/iceberg/overv...
matsonj 1y ago
Sure - https://motherduck.com/blog/15-companies-duckdb-in-prod/ https://motherduck.com/blog/15-companies-duckdb-in-prod/

_ea1k 1y ago

Agreed. I just wish that the vector support were fully supported. It has been experimental for a long time.

geysersam 1y ago

I love duckdb but I don't understand your second point. How does duckdb wasm decrease the latency of network calls in your app?

benrutter 1y ago
Not OP, but depending on your use case, client side operations could be a big deal. Something like a dashboard app for instance, might have callbacks where it needs to run a new SQL query based on input, but be small enough to make loading all data onto menory and querying locally a mich faster option. Obviously ymmv and if you have a dashboard with a heavy data set under the hood, your gonna make a lot of users unhappy fast by doing that.
mritchie712 1y ago
for many uses cases, you can cache a dataset client side that can be re-queried to avoid network calls

adsharma 1y ago

Love the DuckLake spec. Now there is something similar for Graphs. https://adsharma.github.io/graph-archiving/ https://adsharma.github.io/graph-archiving/ https://adsharma.github.io/beating-the-CAP-theorem-for-graphs/ https://adsharma.github.io/beating-the-CAP-theorem-for-graph... The catalog is based on the now archived kuzu graph db project. Development continuing here: https://github.com/LadybugDB/ladybug https://github.com/LadybugDB/ladybug