4 ms·

Show HN: Git-subcopy lets you link files across repositories

110 points by jD91mZM2 7y ago

narnianal 7y ago
What's the architecture of this idea? Is there a reasonable assumption that it would keep the complexity for maintainers low in 3+ repos having a relationship with each other? If so, how?
- jD91mZM2 7y ago
  Not sure I understand the question correctly, but here goes: My personal bias is to keep using submodules or at least subtrees where possible, mainly because it's builtin and because it's written by smart people. This is mostly for when repositories are pretty big and you don't want to submodule/subtree the entirety. But that's just me. I hope the maintainer complexity is as low as any other git solution, all the rebasing tools are just using standard git so there should neither be any advantages or disadvantages. The advantage of using this is to not check in everything into one repository (perhaps avoiding accidental modification of unrelated files that could cause merge conflicts).
  - narnianal 7y ago
    My impression is that monorepo got popular because submodules are too complicated to use on 3+ projects combined together. And I don't mean that the user interface sucks or that it's illogical to use, but most people will tend to use rather less submodules than more, even experienced git users. Therefore I think the breakthrough will come with someone who really understands submodules but has a great idea how to simplify the process. I hope this project may be a step in the right direction. Therefore I want to know more about the internals.
    - jD91mZM2 7y ago
      I'm afraid I'm not the messiah you're looking for, then. This is not a replacement for subtrees/submodules. But from my limited experience I agree with what you're saying :)
apostacy 7y ago
I like this idea and want to know more, but I wish the README.md actually told us how and what, instead of just why. I didn't want to watch an asciicast.
- jD91mZM2 7y ago
  Thank you for the feedback, I'll update the README EDIT: Updated
polyterative 7y ago
Hell I want this to be a standard
jesse_m 7y ago
Is this similar to Android repo tool?
- jD91mZM2 7y ago
  I'm unfamiliar with that tool, so you tell me! Looks like it though, from what I'm reading, with the exception of keeping the internal configuration in a git-friendly format instead of huge XML.
AlotOfReading 7y ago
Every so often I find it necessary to move things from one repo to another and preserve history along the way. I've made do with a janky shell script that recreates all the commits touching relevant files in the new repo, but this looks like a much less sketchy way to do it. It even sends changes back to the original repo if I understand the code right.
- minxomat 7y ago
  Git can already do that with subtree, no other tools needed.
  - jD91mZM2 7y ago
    (Note that subtree is a user-contributed script, but it's been made to ship with most git installations)
  - oefrha 7y ago
    git-subtree is for incorporating an entire repo. To cherry-pick a single directory from another repo you probably need git-filter-branch to split that out into a standalone repo first, followed by git-subtree. Still, this is just two commands so I don’t see why a third party tool is warranted, especially since the third-party tool here seems to require a more complicated process (maybe it achieves more than what I think it achieves, I just don’t see it from the description, especially considering the problem it set out to solve seems to be exactly what I had in mind).
    - AlotOfReading 7y ago
      Filter-branch / subtree doesn't let you move files outside the subdirectory, as far as I'm aware. Happy to be corrected on this point though.
      - js2 7y ago
        There's basically nothing you can't do with filter-branch and the variety of filters it can execute. Moving a subdirectory to the root is trivial with --subdirectory-filter, but even without that, you could do it with --index-filter or --tree-filter.
dmoose 7y ago
I was looking at this for a similar use case https://bit.dev/ https://bit.dev/
jD91mZM2 7y ago
I believe there's some confusion around what git-subcopy does. It doesn't actually preserve history or anything like it. It just clones a certain file into your repository as a new commit, while saving which revision the file is from. A potentially good candidate for using git-subcopy would be on this file: https://github.com/nix-community/rnix-parser/blob/master/benches/all-packages.nix https://github.com/nix-community/rnix-parser/blob/master/ben.... It's copy-pasted from the nixpkgs upstream repository. It can't be used as a submodule - it's just one file out of dozens, no way people want to clone all that. Even worse with subtree, that'll always clone the whole upstream repo, and pollute the history on top of that. But the way it is currently, there's also no hint where it comes from. git-subcopy is basically just me imagining a solution where you can not only link files or directories between repos without bloating down code size or history, but also modify them and later check what modifications you've made and rebase them. If you don't have a use case for this tool, good for you!

110 points by jD91mZM2 7y ago

What's the architecture of this idea? Is there a reasonable assumption that it would keep the complexity for maintainers low in 3+ repos having a relationship with each other? If so, how?

jD91mZM2 7y ago
Not sure I understand the question correctly, but here goes: My personal bias is to keep using submodules or at least subtrees where possible, mainly because it's builtin and because it's written by smart people. This is mostly for when repositories are pretty big and you don't want to submodule/subtree the entirety. But that's just me. I hope the maintainer complexity is as low as any other git solution, all the rebasing tools are just using standard git so there should neither be any advantages or disadvantages. The advantage of using this is to not check in everything into one repository (perhaps avoiding accidental modification of unrelated files that could cause merge conflicts).
- narnianal 7y ago
  My impression is that monorepo got popular because submodules are too complicated to use on 3+ projects combined together. And I don't mean that the user interface sucks or that it's illogical to use, but most people will tend to use rather less submodules than more, even experienced git users. Therefore I think the breakthrough will come with someone who really understands submodules but has a great idea how to simplify the process. I hope this project may be a step in the right direction. Therefore I want to know more about the internals.
  - jD91mZM2 7y ago
    I'm afraid I'm not the messiah you're looking for, then. This is not a replacement for subtrees/submodules. But from my limited experience I agree with what you're saying :)

apostacy 7y ago

I like this idea and want to know more, but I wish the README.md actually told us how and what, instead of just why. I didn't want to watch an asciicast.

jD91mZM2 7y ago
Thank you for the feedback, I'll update the README EDIT: Updated

polyterative 7y ago

Hell I want this to be a standard

jesse_m 7y ago

Is this similar to Android repo tool?

jD91mZM2 7y ago
I'm unfamiliar with that tool, so you tell me! Looks like it though, from what I'm reading, with the exception of keeping the internal configuration in a git-friendly format instead of huge XML.

AlotOfReading 7y ago

Every so often I find it necessary to move things from one repo to another and preserve history along the way. I've made do with a janky shell script that recreates all the commits touching relevant files in the new repo, but this looks like a much less sketchy way to do it. It even sends changes back to the original repo if I understand the code right.

minxomat 7y ago
Git can already do that with subtree, no other tools needed.
- jD91mZM2 7y ago
  (Note that subtree is a user-contributed script, but it's been made to ship with most git installations)
- oefrha 7y ago
  git-subtree is for incorporating an entire repo. To cherry-pick a single directory from another repo you probably need git-filter-branch to split that out into a standalone repo first, followed by git-subtree. Still, this is just two commands so I don’t see why a third party tool is warranted, especially since the third-party tool here seems to require a more complicated process (maybe it achieves more than what I think it achieves, I just don’t see it from the description, especially considering the problem it set out to solve seems to be exactly what I had in mind).
  - AlotOfReading 7y ago
    Filter-branch / subtree doesn't let you move files outside the subdirectory, as far as I'm aware. Happy to be corrected on this point though.
    - js2 7y ago
      There's basically nothing you can't do with filter-branch and the variety of filters it can execute. Moving a subdirectory to the root is trivial with --subdirectory-filter, but even without that, you could do it with --index-filter or --tree-filter.

dmoose 7y ago

I was looking at this for a similar use case https://bit.dev/ https://bit.dev/

jD91mZM2 7y ago

I believe there's some confusion around what git-subcopy does. It doesn't actually preserve history or anything like it. It just clones a certain file into your repository as a new commit, while saving which revision the file is from. A potentially good candidate for using git-subcopy would be on this file: https://github.com/nix-community/rnix-parser/blob/master/benches/all-packages.nix https://github.com/nix-community/rnix-parser/blob/master/ben.... It's copy-pasted from the nixpkgs upstream repository. It can't be used as a submodule - it's just one file out of dozens, no way people want to clone all that. Even worse with subtree, that'll always clone the whole upstream repo, and pollute the history on top of that. But the way it is currently, there's also no hint where it comes from. git-subcopy is basically just me imagining a solution where you can not only link files or directories between repos without bloating down code size or history, but also modify them and later check what modifications you've made and rebase them. If you don't have a use case for this tool, good for you!