3 ms·
A couple of demos: https://fivethirtyeight.datasettes.com/ https://fivethirtyeight.datasettes.com/ contains SQLite versions of every CSV published by the FiveT
by simonw 9y ago
A couple of demos:
https://fivethirtyeight.datasettes.com/ https://fivethirtyeight.datasettes.com/ contains SQLite versions of every CSV published by the FiveThirtyEight crew to https://github.com/fivethirtyeight/data https://github.com/fivethirtyeight/data
I built a separate tool on top of pandas which can mass-convert CSVs to SQLite - https://github.com/simonw/csvs-to-sqlite https://github.com/simonw/csvs-to-sqlite - so building the database for https://fivethirtyeight.datasettes.com/ https://fivethirtyeight.datasettes.com/ is now a one-liner:
csvs-to-sqlite ~/fivethirtyeight/data fivethirtyeight.db
https://parlgov.datasettes.com/ https://parlgov.datasettes.com/ uses the SQLite database from http://www.parlgov.org/ http://www.parlgov.org/ - this is a more interesting demo as it includes things like SQL views - e.g. https://parlgov.datasettes.com/parlgov/view_election https://parlgov.datasettes.com/parlgov/view_election
Both of these demos are served through Cloudflare - since the URL to each database incorporates a sha256 hash of the database contents, I can send a far-future cache expiry header on everything.
- deleted 9y ago[deleted]
- vog 9y ago> since the URL to each database incorporates a sha256 hash of the database contents Small nitpick: It incorporates a SHA-256 prefix, i.e. a greatly reduced SHA-256 hash, so the cryptographic integrity guarantees no longer hold. > send a far-future cache expiry header on everything If you really want to do that, you should include the full SHA-256 hash in the URL. If you want to shorten the URL, use a more compact encoding (e.g. Base64 or Ascii85, hex). Alternatively, make it optional (but I admit this probably defeats the purpose): Short URLs without hash and short expiration, long URLs with hash and eternal expiration.
- simonw 9y agoI don’t need cryptographic guarantees because I’m not dealing with untrusted data: if you want to screw up Datasette’s caching model by deliberately creating two different databases with the same sha256 prefix then go ahead - you’ll only be hurting your own deployment. As such I figured that if a SHA prefix is good enough for common usage of git, it’s good enough for me.