3 ms·
The neatest thing i seen is you can put a sqlite db on a http server and read it effectively using range requests
by cyanydeez 1y ago
The neatest thing i seen is you can put a sqlite db on a http server and read it effectively using range requests
- pmarreck 1y agoso basically using the http server as a randomly-accessed data store? sounds about right
- simlevesque 1y agoIn my experience, this works when the db is read only. And in these read only cases I'd use Parquet files queried with Duckdb Wasm.
- johannes1234321 1y agoThere are implementations for that: For example https://github.com/psanford/sqlite3vfshttp https://github.com/psanford/sqlite3vfshttp or https://github.com/phiresky/sql.js-httpvfs https://github.com/phiresky/sql.js-httpvfs
- ncruces 1y agoThe latency on those requests matters, though. You'll probably benefit from using the largest possible page size; also, keep alive; etc. But even then, you'll pull at most 64 KiB per request. If you managed to have response times of 10 ms, you'd be pulling at most 52 Mbps. So yeah, if your queries end up reading just a couple of pages, it's great. If they require a full table scan, you need some smart prefetching+caching to hide the latency.