3 ms·
I've become a fan of using SQLite-utils to work with CSV or JSON files. It's a two step process though. One to create and insert into a DB and a second to sele
by adamgordonbell 4y ago
I've become a fan of using SQLite-utils to work with CSV or JSON files.
It's a two step process though. One to create and insert into a DB and a second to select from and return.
https://sqlite-utils.datasette.io/en/stable/index.html https://sqlite-utils.datasette.io/en/stable/index.html
- simonw 4y agoI added a feature last year that lets you do this as a one-step process - "sqlite-utils memory": https://simonwillison.net/2021/Jun/19/sqlite-utils-memory/ https://simonwillison.net/2021/Jun/19/sqlite-utils-memory/
- adamgordonbell 4y agoVery cool!
- eatonphil 4y agoThose steps still exist though surely, just managed by the one command. If you're querying via SQLite you have to ingest first.
- simonw 4y agoYup - 'sqlite-utils memory' works by creating an in-memory database, importing various different file formats into it, running the query and then throwing everything away again. It's generally not a good fit for anything over a few dozen MBs of data.