3 ms·

the sqlite RIIR is going quite well: https://turso.tech/blog/beyond-the-single-writer-limitation-with-tursos-concurrent-writes https://turso.tech/blog/beyond-th

by jitl 1y ago

the sqlite RIIR is going quite well: https://turso.tech/blog/beyond-the-single-writer-limitation-with-tursos-concurrent-writes https://turso.tech/blog/beyond-the-single-writer-limitation-... (sqlite is quite a bit smaller than DuckDB tho)

kragen 1y ago
Is it? It's much less new.
- jitl 1y ago
  for one, duckdb includes all of sqlite (and many other dependencies). it knows how to do things like efficiently query over parquet files in s3. it's expansive - a swiss army knife for working with data wherever it's at. sqlite is a "self contained system" depending on no external software except c standard library for target os: > A minimal build of SQLite requires just these routines from the standard C library: > memcmp(), memcpy(), memmove(), memset(), strcmp(), strlen(), strncmp() > Most builds also use the system memory allocation routines: > malloc(), realloc(), free() > Default builds of SQLite contain appropriate VFS objects for talking to the underlying operating system, and those VFS objects will contain operating system calls such as open(), read(), write(), fsync(), and so forth Quoting from the appropriately named https://sqlite.org/selfcontained.html https://sqlite.org/selfcontained.html as a very rough and unfair estimate between the two project's source, sqlite is about 8% the size of duckdb: $ pwd /Users/jitl/src/duckdb/src $ sloc . ---------- Result ------------ Physical : 418092 Source : 317274 Comment : 50113 Single-line comment : 46187 Block comment : 3926 Mixed : 4415 Empty block comment : 588 Empty : 55708 To Do : 136 Number of files read : 2611 ---------------------------- $ cd ~/Downloads/sqlite-amalgamation-3500400/ $ sloc . ---------- Result ------------ Physical : 34742 Source : 25801 Comment : 8110 Single-line comment : 1 Block comment : 8109 Mixed : 1257 Empty block comment : 1 Empty : 2089 To Do : 5 Number of files read : 2 ----------------------------
  - kragen 1y ago
    Oh, wow! I really had no idea!
zX41ZdbW 1y ago
I'm trying to make Turso load some data, but it is so slow that even several months are not enough to load the dataset: https://github.com/ClickHouse/ClickBench/issues/336 https://github.com/ClickHouse/ClickBench/issues/336

by jitl 1y ago

kragen 1y ago

Is it? It's much less new.

jitl 1y ago
for one, duckdb includes all of sqlite (and many other dependencies). it knows how to do things like efficiently query over parquet files in s3. it's expansive - a swiss army knife for working with data wherever it's at. sqlite is a "self contained system" depending on no external software except c standard library for target os: > A minimal build of SQLite requires just these routines from the standard C library: > memcmp(), memcpy(), memmove(), memset(), strcmp(), strlen(), strncmp() > Most builds also use the system memory allocation routines: > malloc(), realloc(), free() > Default builds of SQLite contain appropriate VFS objects for talking to the underlying operating system, and those VFS objects will contain operating system calls such as open(), read(), write(), fsync(), and so forth Quoting from the appropriately named https://sqlite.org/selfcontained.html https://sqlite.org/selfcontained.html as a very rough and unfair estimate between the two project's source, sqlite is about 8% the size of duckdb: $ pwd /Users/jitl/src/duckdb/src $ sloc . ---------- Result ------------ Physical : 418092 Source : 317274 Comment : 50113 Single-line comment : 46187 Block comment : 3926 Mixed : 4415 Empty block comment : 588 Empty : 55708 To Do : 136 Number of files read : 2611 ---------------------------- $ cd ~/Downloads/sqlite-amalgamation-3500400/ $ sloc . ---------- Result ------------ Physical : 34742 Source : 25801 Comment : 8110 Single-line comment : 1 Block comment : 8109 Mixed : 1257 Empty block comment : 1 Empty : 2089 To Do : 5 Number of files read : 2 ----------------------------
- kragen 1y ago
  Oh, wow! I really had no idea!

zX41ZdbW 1y ago

I'm trying to make Turso load some data, but it is so slow that even several months are not enough to load the dataset: https://github.com/ClickHouse/ClickBench/issues/336 https://github.com/ClickHouse/ClickBench/issues/336