7 ms·

The takedown didn't backfire at all just because some people decided to publish on Twitter a tarball that still happens to be hosted in many other places so the

The takedown didn't backfire at all just because some people decided to publish on Twitter a tarball that still happens to be hosted in many other places so there is absolutely no risk of it getting lost. The problem is that the project may not survive this. Major distributions for example will no longer carry the project and likely refuse to touch it even with a ten-foot pole (think media codecs situation). It will be relegated to 3rd party repos. They will lose users, they will lose contributors. And how long until YT's (and/or the other supported websites) HTML changes more rapidly until the remaining manpower can't keep up?

jk700 6y ago
As someone who uses youtube-dl for all the videos online, it's presence in any repos isn't as important as it may seem, because it breaks from time to time for some websites and you need to update it in order to keep using it, which is only easy if you keep it in your home directory so you can just run youtube-dl -U once it breaks.
- AshamedCaptain 6y ago
  I completely disagree. I have been using for example NewPipe from FDroid which is autobuild regularly as native Android program. I would no longer expect FDroid now to keep doing this if they risk a DCMA letter that takes out their entire repo. And I would assume a shitton of users use yt-dl code through another GUI application, rather than directly through an EasyInstall/virtualenv/whatever. And will most py repos still dare to host/link this code if the risk is a DMCA letter to their ISP?
  - cyphar 6y ago
    NewPipe doesn't use youtube-dl, they have their own implementation of the stream extraction code[1]. Unless that codebase also has test cases which specifically try to download RIAA content, they would need to find a different justification for a DMCA takedown. [1]: https://github.com/TeamNewPipe/NewPipeExtractor https://github.com/TeamNewPipe/NewPipeExtractor
    - AshamedCaptain 6y ago
      I realize that since NewPipe's Java; but it's just an example. I really doubt the "it's just the test cases" justification btw, since test cases would likely be fair use. The deencryption stuff is the problem, and NewPipe does it too.
      - deleted 6y ago
        [deleted]
  - toyg 6y ago
    Cam down. This will just end up like DeCSS, when it was packaged separately and left to the user to trigger the necessary download. Distribution/integration is the easy part. The hard part is to anonymise and secure core developers, and to allow contributions to continue flowing in a safe manner.
    - AshamedCaptain 6y ago
      But user loss implies contributor loss, and that is the problem. It's not about me being able to find it (I definitely will be able to :) ), or guaranteeing a "safe harbor" to developers (there are plenty of organizations that are dying to host something like this). It's about a project hat requires a shit ton of constant manpower not having it because of it being "tainted".
      - toyg 6y ago
        It’s not C++ and complex encryption, this is a python web scraper - hardly rocket science. I don’t think manpower will be significantly harder to come by than it was in the past — the opposite, in fact.
        AshamedCaptain 6y ago
        It did not say it requires experienced manpower, I said it requires a lot of it, and importantly, constantly. It's easy to find a bunch of "fanatics" that will rise to the ocasion to "defend their freedoms" right now. But that is not what these projects require. On the other hand, it's hard to have volunteers who will keep doing the _menial_ changes for years as the different websites change their HTML/layout. Anything less and the tool loses a huge chunk of its usefulness.
        toyg 6y ago
        Yes, but this was the same before. If anything, the current visibility is a shot in the arm. I don't think that, once the dust settles, youtube-dl will end up with less regular contributors than it had before - likely the opposite, in fact.
  - efreak 6y ago
    NewPipe has their own fdroid repo already, so that shouldn't cause any serious issues
choppaface 6y ago
At this point it's less about software maintenance and more about the streisand effect ala DeCSS http://www.cs.cmu.edu/~dst/DeCSS/Gallery/ http://www.cs.cmu.edu/~dst/DeCSS/Gallery/
- AshamedCaptain 6y ago
  No, it's not; this misses the point. A tarball of _ancient_ DeCSS code has some value because you can use it to "decss" _all_ content made _before_ a certain date (e.g. most physical DVDs). So it makes sense to preserve the tarball in any method possible. A tarball of an _ancient_ youtube-dl version is absolutely useless because the youtube HTML will have changed a million times in the meanwhile. You will not be able to use it for anything, neither for old nor for new content. Publishing/preserving youtube-dl tarballs is an absolute waste of effort. The RIAA here is targeting existing project developers, and possibly also users. Trying to scare them away. Not trying to censorship old tarballs of the code which will quickly become useless.
  - JdeBP 6y ago
    You are basing your argument on the highly erroneous premise that YouTube is all that youtube-dl is usable for. * https://news.ycombinator.com/item?id=24905640 https://news.ycombinator.com/item?id=24905640
    - AshamedCaptain 6y ago
      No, the argument does not depend on that premise, unless you are suggesting that the other websites never ever ever change their HTML. i.e. replace YT with $YOUR_FAVORITE_VIDEO_HOSTING_WEBSITE in the message above.
      - JdeBP 6y ago
        Your argument that "You will not be able to use it for anything" because "youtube HTML will have changed a million times" is most definitely based upon that erroneous premise. Or a misunderstanding of the word "anything". (-:
wolco2 6y ago
There are smaller projects who get along fine with larger scopes but I think the opposite will hapen. This has given them enough popularity to overtake other similiar projects. This can turn be widely popular. I started downloading from youtube because of this. I haven't come across RIAA protected content but who knows what the future holds
weego 6y ago
It's not worth playing cat and mouse with scrapers, it'll always be relatively trivial to solve compared to the engineering effort to come up with a new strategy, and that's with brute force scraping which is a fair way behind methods commercial scraping companies have been using for a long time.
- deleted 6y ago
  [deleted]
suby 6y ago
I think you might be right, but I also don't know how viable this was before the DMCA. I have never been able to get youtube-dl to work with (both ubuntu / mint) distro packages. The packages always fall out of date due to youtube changing things, and you need to download it directly from the youtube-dl website regardless.
- hsbauauvhabzb 6y ago
  Repo packages moved slowly, pip or building from source was often the only way of reliably getting it to work.
  - deleted 6y ago
    [deleted]
  - AshamedCaptain 6y ago
    Pip is a repo.
    - hsbauauvhabzb 6y ago
      Not in the same light as preconfigured repos in Ubuntu et al. I don’t think pip is even installed by default under most distributions, and Iirc some actively discourage its use. It’s also Supply chain insecure when compared to trusted repos. But congratulations on catching the technicality!
jmnicolas 6y ago
> Major distributions for example will no longer carry the project You had to install the Github version anyway if you wanted to have a recent version that would work with YT.
- AshamedCaptain 6y ago
  (Not here, my distro has a backports repo, and I'd suspect a significant if not majority of users get it from a repo of some sort, distro or not). And even then, where are users going to search the most recent version? You will definitely not preserve the same number of users if you switch from yt-dl being "one apt-get away/one FDroid install away/one pip install away" to "one bittorrent search/wget from shady .ru website". And let's not talk about how will people contribute. The project could be as good as dead.
  - Arnavion 6y ago
    I'm on a rolling distro and I still maintained my own copy of youtube-dl (with a cronjob to run `youtube-dl -U`) instead of using my distro's package. If I used my distro's package, every time it broke I would have to wait a few days for my distro to update its package with the fix, even though upstream would've cut a release with the fix within a few hours.
- bityard 6y ago
  Not at all, I've been installing the latest releases of youtube-dl from PyPI via pip for years. In fact, it's still there. The RIAA takedown has had no effect (so far) on my usage of youtube-dl. The RIAA clearly doesn't have any understanding about how open source Python programs are commonly distributed.
  - mafuy 6y ago
    That's not wrong, but it is missing the point: Ongoing development is impeded by the takedown. That means the tool will not work anymore in the near future, rendering it useless.
  - 6y ago
eikenberry 6y ago
I don't think distributions dropping it will have that big of an impact considering you only ever used those versions to bootstrap as you needed to `-U` update it regularly to keep it working (except maybe on Arch and similar bleeding edge, rolling distros). I kind of hope they adopt flatpak or appimage based disto for it, giving it a more standard distribution mechanism and promoting one of those platforms. Either way, this sort of advertisement is hard to beat. I'd expect the project to come out of this with more manpower, not less.
nocturnial 6y ago
> The problem is that the project may not survive this. I thought all they need to do is to issue a counter-notice. Popcorn did this and got their code reinstated on github: https://news.ycombinator.com/item?id=24885079 https://news.ycombinator.com/item?id=24885079 In the worst case, they need to adjust their tests. This is what the issuer of the complaint brought up. Their tests of the code downloaded a couple of seconds of what the RIAA thought is copyrighted material. If I'm not mistaken, this is what they are basing their case on. If youtube-dl removes those explicit urls from their test cases, I don't think they have a case. You could argue fair use, but remember it's only a test case in that program. The authors could just as easily reprogram the test cases to accept any url a user wants to test. Anyways, consult lawyers, perhaps change the test code and issue a counter notice and move on.
- heavyset_go 6y ago
  It's not a traditional DMCA copyright take down, but a take down that's enforced under a different section of the DMCA. I believe that in this case, there is no avenue for issuing a counter notice, but I'm not a lawyer and I'd be happy to be proven wrong.
  - nocturnial 6y ago
    Github files this notice under their dmca notices[0]. Maybe it's not a legal dmca notice, but I would assume if github files it as such, then the internal rules of github would apply. This includes a counter-notice. If it's not a dmca takedown and those rules don't apply then I would assume github would need to explain why they filed it as such. (It's probably the RIAAs "fault", but "we" have only direct contact with github so we need to go through them) [0] https://github.com/github/dmca/blob/master/2020/10/2020-10-23-RIAA.md https://github.com/github/dmca/blob/master/2020/10/2020-10-2...
  - boomboomsubban 6y ago
    The allegedly relevant sections of the DMCA do not contain any forced takedown measures except under court order. So I agree it doesn't seem to be a traditional takedown order, there doesn't seem to be any other reason they took it down.

jk700 6y ago

As someone who uses youtube-dl for all the videos online, it's presence in any repos isn't as important as it may seem, because it breaks from time to time for some websites and you need to update it in order to keep using it, which is only easy if you keep it in your home directory so you can just run youtube-dl -U once it breaks.

AshamedCaptain 6y ago
I completely disagree. I have been using for example NewPipe from FDroid which is autobuild regularly as native Android program. I would no longer expect FDroid now to keep doing this if they risk a DCMA letter that takes out their entire repo. And I would assume a shitton of users use yt-dl code through another GUI application, rather than directly through an EasyInstall/virtualenv/whatever. And will most py repos still dare to host/link this code if the risk is a DMCA letter to their ISP?
- cyphar 6y ago
  NewPipe doesn't use youtube-dl, they have their own implementation of the stream extraction code[1]. Unless that codebase also has test cases which specifically try to download RIAA content, they would need to find a different justification for a DMCA takedown. [1]: https://github.com/TeamNewPipe/NewPipeExtractor https://github.com/TeamNewPipe/NewPipeExtractor
  - AshamedCaptain 6y ago
    I realize that since NewPipe's Java; but it's just an example. I really doubt the "it's just the test cases" justification btw, since test cases would likely be fair use. The deencryption stuff is the problem, and NewPipe does it too.
    - deleted 6y ago
      [deleted]
- toyg 6y ago
  Cam down. This will just end up like DeCSS, when it was packaged separately and left to the user to trigger the necessary download. Distribution/integration is the easy part. The hard part is to anonymise and secure core developers, and to allow contributions to continue flowing in a safe manner.
  - AshamedCaptain 6y ago
    But user loss implies contributor loss, and that is the problem. It's not about me being able to find it (I definitely will be able to :) ), or guaranteeing a "safe harbor" to developers (there are plenty of organizations that are dying to host something like this). It's about a project hat requires a shit ton of constant manpower not having it because of it being "tainted".
    - toyg 6y ago
      It’s not C++ and complex encryption, this is a python web scraper - hardly rocket science. I don’t think manpower will be significantly harder to come by than it was in the past — the opposite, in fact.
      - AshamedCaptain 6y ago
        It did not say it requires experienced manpower, I said it requires a lot of it, and importantly, constantly. It's easy to find a bunch of "fanatics" that will rise to the ocasion to "defend their freedoms" right now. But that is not what these projects require. On the other hand, it's hard to have volunteers who will keep doing the _menial_ changes for years as the different websites change their HTML/layout. Anything less and the tool loses a huge chunk of its usefulness.
        toyg 6y ago
        Yes, but this was the same before. If anything, the current visibility is a shot in the arm. I don't think that, once the dust settles, youtube-dl will end up with less regular contributors than it had before - likely the opposite, in fact.
- efreak 6y ago
  NewPipe has their own fdroid repo already, so that shouldn't cause any serious issues

choppaface 6y ago

At this point it's less about software maintenance and more about the streisand effect ala DeCSS http://www.cs.cmu.edu/~dst/DeCSS/Gallery/ http://www.cs.cmu.edu/~dst/DeCSS/Gallery/

AshamedCaptain 6y ago
No, it's not; this misses the point. A tarball of _ancient_ DeCSS code has some value because you can use it to "decss" _all_ content made _before_ a certain date (e.g. most physical DVDs). So it makes sense to preserve the tarball in any method possible. A tarball of an _ancient_ youtube-dl version is absolutely useless because the youtube HTML will have changed a million times in the meanwhile. You will not be able to use it for anything, neither for old nor for new content. Publishing/preserving youtube-dl tarballs is an absolute waste of effort. The RIAA here is targeting existing project developers, and possibly also users. Trying to scare them away. Not trying to censorship old tarballs of the code which will quickly become useless.
- JdeBP 6y ago
  You are basing your argument on the highly erroneous premise that YouTube is all that youtube-dl is usable for. * https://news.ycombinator.com/item?id=24905640 https://news.ycombinator.com/item?id=24905640
  - AshamedCaptain 6y ago
    No, the argument does not depend on that premise, unless you are suggesting that the other websites never ever ever change their HTML. i.e. replace YT with $YOUR_FAVORITE_VIDEO_HOSTING_WEBSITE in the message above.
    - JdeBP 6y ago
      Your argument that "You will not be able to use it for anything" because "youtube HTML will have changed a million times" is most definitely based upon that erroneous premise. Or a misunderstanding of the word "anything". (-:

wolco2 6y ago

There are smaller projects who get along fine with larger scopes but I think the opposite will hapen. This has given them enough popularity to overtake other similiar projects. This can turn be widely popular. I started downloading from youtube because of this. I haven't come across RIAA protected content but who knows what the future holds

weego 6y ago

It's not worth playing cat and mouse with scrapers, it'll always be relatively trivial to solve compared to the engineering effort to come up with a new strategy, and that's with brute force scraping which is a fair way behind methods commercial scraping companies have been using for a long time.

deleted 6y ago
[deleted]

suby 6y ago

I think you might be right, but I also don't know how viable this was before the DMCA. I have never been able to get youtube-dl to work with (both ubuntu / mint) distro packages. The packages always fall out of date due to youtube changing things, and you need to download it directly from the youtube-dl website regardless.

hsbauauvhabzb 6y ago
Repo packages moved slowly, pip or building from source was often the only way of reliably getting it to work.
- deleted 6y ago
  [deleted]
- AshamedCaptain 6y ago
  Pip is a repo.
  - hsbauauvhabzb 6y ago
    Not in the same light as preconfigured repos in Ubuntu et al. I don’t think pip is even installed by default under most distributions, and Iirc some actively discourage its use. It’s also Supply chain insecure when compared to trusted repos. But congratulations on catching the technicality!

jmnicolas 6y ago

> Major distributions for example will no longer carry the project You had to install the Github version anyway if you wanted to have a recent version that would work with YT.

AshamedCaptain 6y ago
(Not here, my distro has a backports repo, and I'd suspect a significant if not majority of users get it from a repo of some sort, distro or not). And even then, where are users going to search the most recent version? You will definitely not preserve the same number of users if you switch from yt-dl being "one apt-get away/one FDroid install away/one pip install away" to "one bittorrent search/wget from shady .ru website". And let's not talk about how will people contribute. The project could be as good as dead.
- Arnavion 6y ago
  I'm on a rolling distro and I still maintained my own copy of youtube-dl (with a cronjob to run `youtube-dl -U`) instead of using my distro's package. If I used my distro's package, every time it broke I would have to wait a few days for my distro to update its package with the fix, even though upstream would've cut a release with the fix within a few hours.
bityard 6y ago
Not at all, I've been installing the latest releases of youtube-dl from PyPI via pip for years. In fact, it's still there. The RIAA takedown has had no effect (so far) on my usage of youtube-dl. The RIAA clearly doesn't have any understanding about how open source Python programs are commonly distributed.
- mafuy 6y ago
  That's not wrong, but it is missing the point: Ongoing development is impeded by the takedown. That means the tool will not work anymore in the near future, rendering it useless.
- 6y ago

eikenberry 6y ago

I don't think distributions dropping it will have that big of an impact considering you only ever used those versions to bootstrap as you needed to `-U` update it regularly to keep it working (except maybe on Arch and similar bleeding edge, rolling distros). I kind of hope they adopt flatpak or appimage based disto for it, giving it a more standard distribution mechanism and promoting one of those platforms. Either way, this sort of advertisement is hard to beat. I'd expect the project to come out of this with more manpower, not less.

nocturnial 6y ago

> The problem is that the project may not survive this. I thought all they need to do is to issue a counter-notice. Popcorn did this and got their code reinstated on github: https://news.ycombinator.com/item?id=24885079 https://news.ycombinator.com/item?id=24885079 In the worst case, they need to adjust their tests. This is what the issuer of the complaint brought up. Their tests of the code downloaded a couple of seconds of what the RIAA thought is copyrighted material. If I'm not mistaken, this is what they are basing their case on. If youtube-dl removes those explicit urls from their test cases, I don't think they have a case. You could argue fair use, but remember it's only a test case in that program. The authors could just as easily reprogram the test cases to accept any url a user wants to test. Anyways, consult lawyers, perhaps change the test code and issue a counter notice and move on.

heavyset_go 6y ago
It's not a traditional DMCA copyright take down, but a take down that's enforced under a different section of the DMCA. I believe that in this case, there is no avenue for issuing a counter notice, but I'm not a lawyer and I'd be happy to be proven wrong.
- nocturnial 6y ago
  Github files this notice under their dmca notices[0]. Maybe it's not a legal dmca notice, but I would assume if github files it as such, then the internal rules of github would apply. This includes a counter-notice. If it's not a dmca takedown and those rules don't apply then I would assume github would need to explain why they filed it as such. (It's probably the RIAAs "fault", but "we" have only direct contact with github so we need to go through them) [0] https://github.com/github/dmca/blob/master/2020/10/2020-10-23-RIAA.md https://github.com/github/dmca/blob/master/2020/10/2020-10-2...
- boomboomsubban 6y ago
  The allegedly relevant sections of the DMCA do not contain any forced takedown measures except under court order. So I agree it doesn't seem to be a traditional takedown order, there doesn't seem to be any other reason they took it down.