2 ms·
Have you considered loading this stuff into SQLite in addition to keeping the raw exports in files on disk? Being able to query your personal data with SQL can
by simonw 6y ago
Have you considered loading this stuff into SQLite in addition to keeping the raw exports in files on disk?
Being able to query your personal data with SQL can get really interesting. I've been using it for my own version of a personal data warehouse, described here: https://simonwillison.net/2020/Nov/14/personal-data-warehouses/ https://simonwillison.net/2020/Nov/14/personal-data-warehous...
- karlicoss 6y agoYep! In fact I've tried interoperating with Datasette (e.g. shared here https://news.ycombinator.com/item?id=25090643 https://news.ycombinator.com/item?id=25090643 ) One secret sauce is using 'automatic' caching of data in sqlite -- this allows both for faster access and having an additional interface for the data as a collateral https://github.com/karlicoss/cachew#readme https://github.com/karlicoss/cachew#readme Still need to polish this a bit, but ultimately hoping to properly plug into Datasette, I'm impressed by its data exploration capabilities!