5 ms·

I read the post twice but don't quite get what the author wants to share. Is it some git(hub) trick I don't understand, just a way of saying Github's/Microsoft'

by Aachen 6y ago

I read the post twice but don't quite get what the author wants to share. Is it some git(hub) trick I don't understand, just a way of saying Github's/Microsoft's monopoly is bad, or something else altogether?

Fellshard 6y ago
They're using the fact the YouTube-DL (sp?) is being removed by GitHub automatically when rehosted in new repositories, due to DMCA complaint (and likely overreach). If someone rehosts his repository onto GitHub, the repository will be taken down, and the rehosting account likely banned.
- akerl_ 6y ago
  Is there any evidence that GitHub is automatically removing reheated copies? Last I saw, GitHub had posted a comment asking folks not to re-host, at risk of being suspended, but there was no indication they’d started any kind of automated scanning. Beyond that, it seems like a very community-hostile move to intentionally embed commit hashes that would even potentially cause a user to get banned from another platform if they push a copy of the code there. This doesn’t stick it to GitHub, it sticks it to the users of your repo.
  - dx034 6y ago
    It'll be a question of how many reposts they see. If enough people keep reposting with throwaway accounts, they will have no other choice but to start scanning automatically.
  - viraptor 6y ago
    It depends if you want to be nice to everyone, it burn some bridges to make a point. The point being that there shouldn't be censoring of specific hashes. Honestly if a user did get banned for sharing that poisoned repo, I would still put 95% responsibility for that on GitHub. But it's not happening yet as far as I know.
    - deleted 6y ago
      [deleted]
- dheera 6y ago
  ROT13 the entire repo? If they figure it out, then ROT14 it. If they figure that out, encrypt it and post the key somewhere that blocks the offices of Github. If they figure that out, we'll still find a way to build a better mouse.
  - nitrogen 6y ago
    If someone wants you to leave, it's usually best to leave. Good riddance to them if you find yourself in that position. Far better would be to spend your efforts finding which RIAA members pressed the RIAA to make this takedown. Follow the money, cui bono, etc.
    - dheera 6y ago
      Why is it best to leave? Protests are a good thing, sometimes, and making sure it is on GitHub in some form or another is a form of such protest. At some point the relevant parties will have to realize that this mouse of a million engineers will always outsmart them. At some point they will have to realize that advocating for FOSS and DRM at the same time is hypocritical. Rosa Parks was asked to leave, but she didn't leave.
      - nitrogen 6y ago
        GitHub is not the instigator, as far as we know (MS is apparently a member of the RIAA though). I am as amused by the clever ways to get youtube-dl back onto GitHub as anyone else, but it's protesting the wrong thing in the wrong place. The problem lies with whichever member organizations inside the RIAA (or nonmember orgs friendly with the RIAA) convinced them to file the takedown. Most of us aren't positioned to find that out, but that's where the most can be gained. Maybe the EFF or similar orgs can help. Then we'll know what the true motives were, and how to address them.
        dheera 6y ago
        At the same time, technological measures can show it to them that they can't feasibly take it down and that they're fighting a losing battle. At some point the RIAA will just have to accept defeat because honestly it's not that much work to defeat their DMCA takedowns. Right now they're issuing what, 1 takedown notice to GitHub? When we make that 100,000 takedown notices, some in ROT13, some in ROT24 on an Israeli server, some in RSA encryption on a Chinese server with keys published on Russian servers, maybe they'll just be too overwhelmed with paperwork at that point.
BoorishBears 6y ago
It's a trick to (theoretically) get someone banned for pushing a cloned project to Github despite not actually containing banned content.
renewiltord 6y ago
Github complies with the DMCA, recently famously through the DMCA takedown of youtube-dl. The writer, Joey Hess (a relatively well-known OSS programmer, moreutils, for instance), is choosing to interpret this as their banning any repo that hosts the commit hash for youtube-dl (which they banned). So he is jokingly showing you a way to create a Github poison pill. The joke is that if you include this 'illegal' hash you subject the repo to being DMCA'd out (not really, but it's sort of an 'imagine if'). Which means his repo is now unpublishable on Github (for long anyway). It's kind of hacker humour in the sort of vein that you'd expect from an old-schooler like him. You sort of have to get the joke to get the joke.
- syshum 6y ago
  is it a joke though.....
  - o-__-o 6y ago
    ..... yes
- Wowfunhappy 6y ago
  I'd find the joke funny if it actually worked. Since as far as I can tell, Github isn't actually banning specific hashes (because they care about the content, not the hashes), and there's no sign that they're going to do so in the future, I can't say I understand the humor...
  - geofft 6y ago
    That was my initial reaction too, but thinking about it, it's potentially (probably not, but potentially) a violation of the ToS even if it's not automatically banned. So, the hacker humor point is that you have a "poison" repo which, if you push it to GitHub, might get your account deleted in the future. You have no guarantee that it won't, for sure.
  - DarkWiiPlayer 6y ago
    The way I read it, it seems to claim that github will just "insert" a known object with that hash into the new repository when it's pushed, though I'm somewhat confused as to how this would work, or how the submodule trick even adds the hash to the commit log in the first place. EDIT: I read the article again and couldn't find any of that, so I must have read that in a comment here on HN instead.
AceJohnny2 6y ago
It's a way to sneak in the hash of a commit (the latest youtube-DL commit) in a non-obvious way. A git repo is fundamentally composed of 3 object types: * Blob (effectively a file) * Tree (which refers to collection of Blobs and Trees, effectively a dir) * Commit (comment/ author info and various other metadata, a Tree, and a previous Commit) This is what the git tools are used to dealing with. However git submodules changed things. With submodules, a Tree object can also contain a commit object! Together with the .gitmodules metadata file, this allows you to include another git repo inside your repo. Joey leveraged that ability to add the youtube-dl latest master commit into his repo as a submodule, but deleted the .gitmodules file so that git wouldn't be verbose about it. And that's how you sneak a commit into a repo. This additionally leverages GitHub's data model (which is really the git data model), where they just have this huge DB of all the objects of all the repos on GitHub. So effectively by including this commit in a repo on GH, you're making it refer to the the entire (buried) youtube-dl repo, but you need to be sneaky to be able to see it.
- _ikke_ 6y ago
  Note that it's only a _reference_ to a commit, not the commit object itself. So from a a github point of view, it would not even make sense to scan / trigger on this. As long as there are no commit objects with that hash, any reference is harmless.
geofft 6y ago
It's Git performance art/protest art, more or less. Git has a mechanism called "submodules" which allows one repo to reference another repo. It doesn't actually include any of the content from the second repo: all that's included is the URL (in the file .gitmodules) and a commit hash (in Git's view of the directory structure). When you clone a Git repository with submodules and you pass an option to git clone, or if you run a git submodule command later, Git will make a nested clone at whatever subdirectory path contains the submodule, and check out that commit. If you make commits in the submodule (and, hopefully, remember to push them), the outer repo will appear to be modified, and you can git add the changes, which will record the new commit hash, but only the hash. The purpose is to deal with vendoring or sometimes making changes to large repositories owned by someone else without copying the whole source code into your own repository. (It's also occasionally used as a mechanism to split up very large repositories that are owned by the same group/organization, e.g., if you have a test suite or large data files or something, or a shared library used by multiple projects, you might find it helpful to use submodules.) https://github.com/ceph/ceph https://github.com/ceph/ceph is an example repo that uses them - "ceph-object-corpus" and "ceph-erasure-code-corpus" at the top level are submodules, and if you click on the .gitmodules file, you'll find that there are other submodules inside src/, too. This is also an example of how GitHub handles submodules also hosted on GitHub - it will link you to the submodule at that commit. So, GitHub does at least some parsing of submodules. The author is claiming that if you include a banned commit hash as a submodule in the history of your own project, and then promptly delete it, it won't noticeably increase the size of your Git repo, nor will you actually include any of the banned content yourself, but GitHub will potentially prevent that history from being pushed. As a result, you have a Git repo that GitHub would not accept. Since the author both does not use GitHub and wants to encourage others to do the same, this would be a fairly effective way of forcing that. (The article doesn't show GitHub actually refusing to accept the history containing this submodule, though - it only shows the submodule having been created, locally. Although maybe the author's argument is simply that pushing it violates the Terms of Service even if it's not blocked by automated means.)

by Aachen 6y ago

Fellshard 6y ago

They're using the fact the YouTube-DL (sp?) is being removed by GitHub automatically when rehosted in new repositories, due to DMCA complaint (and likely overreach). If someone rehosts his repository onto GitHub, the repository will be taken down, and the rehosting account likely banned.

akerl_ 6y ago
Is there any evidence that GitHub is automatically removing reheated copies? Last I saw, GitHub had posted a comment asking folks not to re-host, at risk of being suspended, but there was no indication they’d started any kind of automated scanning. Beyond that, it seems like a very community-hostile move to intentionally embed commit hashes that would even potentially cause a user to get banned from another platform if they push a copy of the code there. This doesn’t stick it to GitHub, it sticks it to the users of your repo.
- dx034 6y ago
  It'll be a question of how many reposts they see. If enough people keep reposting with throwaway accounts, they will have no other choice but to start scanning automatically.
- viraptor 6y ago
  It depends if you want to be nice to everyone, it burn some bridges to make a point. The point being that there shouldn't be censoring of specific hashes. Honestly if a user did get banned for sharing that poisoned repo, I would still put 95% responsibility for that on GitHub. But it's not happening yet as far as I know.
  - deleted 6y ago
    [deleted]
dheera 6y ago
ROT13 the entire repo? If they figure it out, then ROT14 it. If they figure that out, encrypt it and post the key somewhere that blocks the offices of Github. If they figure that out, we'll still find a way to build a better mouse.
- nitrogen 6y ago
  If someone wants you to leave, it's usually best to leave. Good riddance to them if you find yourself in that position. Far better would be to spend your efforts finding which RIAA members pressed the RIAA to make this takedown. Follow the money, cui bono, etc.
  - dheera 6y ago
    Why is it best to leave? Protests are a good thing, sometimes, and making sure it is on GitHub in some form or another is a form of such protest. At some point the relevant parties will have to realize that this mouse of a million engineers will always outsmart them. At some point they will have to realize that advocating for FOSS and DRM at the same time is hypocritical. Rosa Parks was asked to leave, but she didn't leave.
    - nitrogen 6y ago
      GitHub is not the instigator, as far as we know (MS is apparently a member of the RIAA though). I am as amused by the clever ways to get youtube-dl back onto GitHub as anyone else, but it's protesting the wrong thing in the wrong place. The problem lies with whichever member organizations inside the RIAA (or nonmember orgs friendly with the RIAA) convinced them to file the takedown. Most of us aren't positioned to find that out, but that's where the most can be gained. Maybe the EFF or similar orgs can help. Then we'll know what the true motives were, and how to address them.
      - dheera 6y ago
        At the same time, technological measures can show it to them that they can't feasibly take it down and that they're fighting a losing battle. At some point the RIAA will just have to accept defeat because honestly it's not that much work to defeat their DMCA takedowns. Right now they're issuing what, 1 takedown notice to GitHub? When we make that 100,000 takedown notices, some in ROT13, some in ROT24 on an Israeli server, some in RSA encryption on a Chinese server with keys published on Russian servers, maybe they'll just be too overwhelmed with paperwork at that point.

BoorishBears 6y ago

It's a trick to (theoretically) get someone banned for pushing a cloned project to Github despite not actually containing banned content.

renewiltord 6y ago

Github complies with the DMCA, recently famously through the DMCA takedown of youtube-dl. The writer, Joey Hess (a relatively well-known OSS programmer, moreutils, for instance), is choosing to interpret this as their banning any repo that hosts the commit hash for youtube-dl (which they banned). So he is jokingly showing you a way to create a Github poison pill. The joke is that if you include this 'illegal' hash you subject the repo to being DMCA'd out (not really, but it's sort of an 'imagine if'). Which means his repo is now unpublishable on Github (for long anyway). It's kind of hacker humour in the sort of vein that you'd expect from an old-schooler like him. You sort of have to get the joke to get the joke.

syshum 6y ago
is it a joke though.....
- o-__-o 6y ago
  ..... yes
Wowfunhappy 6y ago
I'd find the joke funny if it actually worked. Since as far as I can tell, Github isn't actually banning specific hashes (because they care about the content, not the hashes), and there's no sign that they're going to do so in the future, I can't say I understand the humor...
- geofft 6y ago
  That was my initial reaction too, but thinking about it, it's potentially (probably not, but potentially) a violation of the ToS even if it's not automatically banned. So, the hacker humor point is that you have a "poison" repo which, if you push it to GitHub, might get your account deleted in the future. You have no guarantee that it won't, for sure.
- DarkWiiPlayer 6y ago
  The way I read it, it seems to claim that github will just "insert" a known object with that hash into the new repository when it's pushed, though I'm somewhat confused as to how this would work, or how the submodule trick even adds the hash to the commit log in the first place. EDIT: I read the article again and couldn't find any of that, so I must have read that in a comment here on HN instead.

AceJohnny2 6y ago

It's a way to sneak in the hash of a commit (the latest youtube-DL commit) in a non-obvious way. A git repo is fundamentally composed of 3 object types: * Blob (effectively a file) * Tree (which refers to collection of Blobs and Trees, effectively a dir) * Commit (comment/ author info and various other metadata, a Tree, and a previous Commit) This is what the git tools are used to dealing with. However git submodules changed things. With submodules, a Tree object can also contain a commit object! Together with the .gitmodules metadata file, this allows you to include another git repo inside your repo. Joey leveraged that ability to add the youtube-dl latest master commit into his repo as a submodule, but deleted the .gitmodules file so that git wouldn't be verbose about it. And that's how you sneak a commit into a repo. This additionally leverages GitHub's data model (which is really the git data model), where they just have this huge DB of all the objects of all the repos on GitHub. So effectively by including this commit in a repo on GH, you're making it refer to the the entire (buried) youtube-dl repo, but you need to be sneaky to be able to see it.

_ikke_ 6y ago
Note that it's only a _reference_ to a commit, not the commit object itself. So from a a github point of view, it would not even make sense to scan / trigger on this. As long as there are no commit objects with that hash, any reference is harmless.

geofft 6y ago

It's Git performance art/protest art, more or less. Git has a mechanism called "submodules" which allows one repo to reference another repo. It doesn't actually include any of the content from the second repo: all that's included is the URL (in the file .gitmodules) and a commit hash (in Git's view of the directory structure). When you clone a Git repository with submodules and you pass an option to git clone, or if you run a git submodule command later, Git will make a nested clone at whatever subdirectory path contains the submodule, and check out that commit. If you make commits in the submodule (and, hopefully, remember to push them), the outer repo will appear to be modified, and you can git add the changes, which will record the new commit hash, but only the hash. The purpose is to deal with vendoring or sometimes making changes to large repositories owned by someone else without copying the whole source code into your own repository. (It's also occasionally used as a mechanism to split up very large repositories that are owned by the same group/organization, e.g., if you have a test suite or large data files or something, or a shared library used by multiple projects, you might find it helpful to use submodules.) https://github.com/ceph/ceph https://github.com/ceph/ceph is an example repo that uses them - "ceph-object-corpus" and "ceph-erasure-code-corpus" at the top level are submodules, and if you click on the .gitmodules file, you'll find that there are other submodules inside src/, too. This is also an example of how GitHub handles submodules also hosted on GitHub - it will link you to the submodule at that commit. So, GitHub does at least some parsing of submodules. The author is claiming that if you include a banned commit hash as a submodule in the history of your own project, and then promptly delete it, it won't noticeably increase the size of your Git repo, nor will you actually include any of the banned content yourself, but GitHub will potentially prevent that history from being pushed. As a result, you have a Git repo that GitHub would not accept. Since the author both does not use GitHub and wants to encourage others to do the same, this would be a fairly effective way of forcing that. (The article doesn't show GitHub actually refusing to accept the history containing this submodule, though - it only shows the submodule having been created, locally. Although maybe the author's argument is simply that pushing it violates the Terms of Service even if it's not blocked by automated means.)