3 ms·
When I helped to take Zulip open-source in 2015, I wrote a simple script that scrubbed secrets from the commit history using git fast-export and git fast-import
by anderskaseorg 6y ago
When I helped to take Zulip open-source in 2015, I wrote a simple script that scrubbed secrets from the commit history using git fast-export and git fast-import. We replaced all our secrets with xxxxxxx placeholders, replaced internal customer references with dummy names, deleted and renamed certain files, and even did some code replacements that caused certain commit diffs to become empty so those commits could be removed from the history.
https://github.com/zulip/zulip/blob/3.3/tools/zanitizer https://github.com/zulip/zulip/blob/3.3/tools/zanitizer
https://github.com/zulip/zulip/blob/3.3/tools/zanitizer_config.pm.sample https://github.com/zulip/zulip/blob/3.3/tools/zanitizer_conf...
The script was really fast (all ~10000 commits in a few minutes), which allowed us to iterate quickly on its configuration as we audited using gitk and other tools for remaining items to scrub.
Doing this work allowed us to release with an essentially complete history going back to the first commit in 2012, which has been a really valuable resource for understanding why various Zulip subsystems were written the way they were.
Nowadays there are other tools for scrubbing history that might be more polished, like BFG: https://rtyley.github.io/bfg-repo-cleaner/ https://rtyley.github.io/bfg-repo-cleaner/
- amichal 6y agoNice tooling. I've used bfg when we knew what patterns to look for. This project didn't generally access private data, had a reasonably well behaved team for most of its life (the pre-linter & code-review commits were my own damn fault). Since it was low risk, I just did a few manual `git log -S ...` and moved on. I was still very happy to have github catch my throwaway credentials and remind me in the most obvious way that these things go in `ENV` and not IN code even in examples!