4 ms·

Reposurgeon was a pretty enticing option for us, as we're currently in the migration of SVN to Git, and the string of tools we have is kind of complicated to ex

Reposurgeon was a pretty enticing option for us, as we're currently in the migration of SVN to Git, and the string of tools we have is kind of complicated to explain to newcomers. Currently the workflow is svn-all-fast-export (SVN to Git) -> git filter-repo (trim content from the repository) -> Git LFS (store large files out-of-band), where each stage has to be manually tested and written, and then hopefully put into a shell script or Bash history or something. Unfortunately, I have yet to get Reposurgeon working for us at all. The documentation was inaccurate, in that it referred to things which only existed in the Go version, and the Go version went OOM (on a server with 384 GB of RAM) on every repository I've tried it on. This includes just reading from an existing, pruned svndump file, so it's not resource contention either. Basically, the tool is great conceptually, but it absolutely does not work for our use case. We had to stick with our existing, messy, multi-component solution, which has worked surprisingly well over the last two years, and since I'm the one doing most of the processing, having kind of a messy system is relatively acceptable for the time being.

db48x 5y ago
We’d be interested in hearing more about your situation, so that we can further improve Reposurgeon. If you can profile it as it reads your svn dump (using the `profile start` and `profile save` commands), it could really help us out. How many commits does your repository have? The largest one we’ve successfully converted from SVN to Git ourselves was 287k commits.
- danudey 5y ago
  Sorry for the late reply, but: 1. I'll try doing some profiling and see what happens 2. The latest repository I tried to import has 671,190 revisions
  - db48x 5y ago
    Wow. That is a huge repository. You can also force it to write all file content (blobs) out to temporary files rather than keeping them in memory by reading the stream from standard input: reposurgeon "read -" … <dump.svn That will be slower and use more disk space, but maybe it will fit.
    - danudey 5y ago
      Thanks for the suggestions, I'll give that a try! And yes, some of our repositories are huge. This one is about 240 GB, and comprises multiple projects/products. Our "most important" repository is only 313,000 revisions, but those revisions take up 93 GB, so it's a not-insubstantial repository as well.

db48x 5y ago

We’d be interested in hearing more about your situation, so that we can further improve Reposurgeon. If you can profile it as it reads your svn dump (using the `profile start` and `profile save` commands), it could really help us out. How many commits does your repository have? The largest one we’ve successfully converted from SVN to Git ourselves was 287k commits.

danudey 5y ago
Sorry for the late reply, but: 1. I'll try doing some profiling and see what happens 2. The latest repository I tried to import has 671,190 revisions
- db48x 5y ago
  Wow. That is a huge repository. You can also force it to write all file content (blobs) out to temporary files rather than keeping them in memory by reading the stream from standard input: reposurgeon "read -" … <dump.svn That will be slower and use more disk space, but maybe it will fit.
  - danudey 5y ago
    Thanks for the suggestions, I'll give that a try! And yes, some of our repositories are huge. This one is about 240 GB, and comprises multiple projects/products. Our "most important" repository is only 313,000 revisions, but those revisions take up 93 GB, so it's a not-insubstantial repository as well.