3 ms·
I love the idea of syncing git commit history with data change history, like using git for a repo of data. It's actually quite possible if you use pretty printe
by keepamovin 3y ago
I love the idea of syncing git commit history with data change history, like using git for a repo of data. It's actually quite possible if you use pretty printed JSON as a record format (or other simple linear text formats).
I explored more of this "git as DB backend" in some places including: https://github.com/dosyago/sirdb https://github.com/dosyago/sirdb
Also, just as a headsup to any folks, the SF version of this "tree map" (Heh) is at: https://web.archive.org/web/20230328192805/https://bsm.sfdpw.org/urbanforestry/ https://web.archive.org/web/20230328192805/https://bsm.sfdpw... (O site and archive seems to be down)
- simonw 3y agoYeah I have a bunch of these using pretty-printed JSON - here's one that scrapes Hacker News for mentions of my site, for example: https://github.com/simonw/scrape-hacker-news-by-domain/blob/main/simonwillison-net.json https://github.com/simonw/scrape-hacker-news-by-domain/blob/...
- keepamovin 3y agoWow you're the creator of datasette! Cool, man! I thought that was a really revolutionary idea, and also related to this notion of git backed DB...I'm so glad to sort of see it confirmed with this! Haha :)
- tizzy 3y agoI've been thinking about this recently also. I can't think of any downsides off the top of my head but I might be blinded by the idea? Can you?
- simonw 3y agoI've had dozens of repos doing this kind of thing for a few years now and I've not found any downsides yet - it just works. There's a size limit to consider: GitHub won't let you push a file larger than 100MB and prefer you to keep the repo itself below 2GB. Most of my scraping projects are a tiny fraction of that size though.