4 ms·
Not too long ago I started compressing statistics data I was collecting from GitHub[1] with ZStandard after noticing that GitHub Actions CI caching used it. It
by andrewmcwatters 4y ago
Not too long ago I started compressing statistics data I was collecting from GitHub[1] with ZStandard after noticing that GitHub Actions CI caching used it.
It’s really remarkable for my use case. I compress gigabyte-sized SQLite databases under the GitHub 100 MB file limit since Git LFS is a non-starter. My tables are collections of follower and stargazer counts to measure user and repository popularity and understand where one might be on a distribution compared to a subset of other GitHub data points.
[1]: https://github.com/andrewmcwattersandco/github-statistics https://github.com/andrewmcwattersandco/github-statistics
- _0w8t 4y agogzip has rsync-friendly mode allowing for much smaller deltas between files during rsync transfer or backups that use rsync-compatible algorithms. No other compressor implements that as far as I know.