7 ms·

Isn’t youtube-dl already updated all the time to keep up with scraping changes? I don’t understand why this would claim to be more maintained?

capableweb 6y ago
Seems to have been originally spawned from this discussion: "Youtube-dl is dying?" https://github.com/ytdl-org/youtube-dl/issues/26462 https://github.com/ytdl-org/youtube-dl/issues/26462 citing many PRs are not getting merged. Anecdote: youtube-dl still works for all the sites I'm using it for.
- BiteCode_dev 6y ago
  I maintain several open source projects. Sometimes PR are not merged for years. I'm not payed for this, I'm not even thanked for this most of the time, so I'm certainly not going to set any deadline. But forking might be a solution, if you can give the necessary horse power for the long run. Most can't.
  - claudeganon 6y ago
    It’s odd to me how nontechnical people have migrated to GitHub to make bug reports on software they use. The other day, I was trying to use a Python library to scrape some stuff. It kept breaking. I looked on the issues page and there were many, essentially useless bug reports asking them to fix the issue. I opened up the script and changed about 7 lines and it worked fine.
    - BiteCode_dev 6y ago
      Well, they look for a channel to communicate their frustration. Most are closed. This one is open. The water follows the path of least resistance.
    - chucky 6y ago
      There are two factors at play here I think. The first is that Github has become quite popular. The second is that Github has worked a lot on usability - it's now very easy for users who are not very technical to create an account and report issues. At times I imagine there could be some kind of "I know how to report bugs"-certification that people could somehow earn. Projects could then use this as a barrier to prevent getting too many useless issue reports. The problem really goes both ways - as a software dev I'm disincentivized to report detailed bugs to projects, because there's a high risk it will get drowned out in the noise. Some projects have quite specific templates for reporting issues. This helps a bit, but you still need someone to go in and remove issues that don't match the template, and few people want to volunteer for that work.
      - claudeganon 6y ago
        Yeah, I guess it’s just an entirely different kind of engagement. I mostly use GitHub to find off the shelf solutions to whatever problem I’m having or to avoid doing the work myself. If I run into any problems I have with something from there, I either try and fix it by tweaking the repository, using another, or just building the thing myself. I think the last time I reported an issue on an external repository was because of something I was made to use at work.
    - OzzyB 6y ago
      I've noticed this trend too -- but I've also noticed people starting GitHub Repos for websites/apps just for the Issues. No code, no PRs, just a way for the public to create Issues. Not sure what this means for the "culture of code" other than things are blurring...
      - claudeganon 6y ago
        I think a lot of it is maybe bleed over from gaming communities, especially emulators, maybe rom hacks too? I’ve noticed barrages of useless bug reports on those repositories. But I’ve been surprised to find them on far afield stuff as well.
      - theidel 6y ago
        At least there is a feature called 'discussions' coming soon to github. That should hopefully minimize the abuse of issues. But that is also due to the fact of current github. Not everyone is displaying their email and there is no other communication channel.
- dependenttypes 6y ago
  I find it gross that dstftw (the core youtubedl maintainer) censored the part about youtube-dlc from that issue.
  - theon144 6y ago
    CTRL+F'ing `youtube-dlc` comes up with several hits - was there more that is missing now?
    - dependenttypes 6y ago
      Check the edit history of the first post.
    - zo1 6y ago
      You can look at the edit history of that comment. You'll see that dstftw edited it to remove references to the youtube_dlc repo. https://github.com/ytdl-org/youtube-dl/issues/26462#issue-688126722 https://github.com/ytdl-org/youtube-dl/issues/26462#issue-68... Unfortunately, can't find a way to link to the edit in question directly. You'll have to click once there to see the diff.
      - saagarjha 6y ago
        Being able to edit people’s comments on GitHub is nice as a maintainer but removing information and completely changing people’s comments is really nasty. Any such effort should be clearly justified, preferably with a visual “edit message” that goes along with the smaller “edited” text.
        u801e 6y ago
        I don't think that editing comments that you didn't make should ever be a feature. This is something that could easily be abused to make it appear that someone else said something they didn't actually way.
        6y ago
- theon144 6y ago
  Counter-anecdote: Even YouTube support is somewhat flaky (age-restricted videos, private (as in, locked behind auth) videos, random failures), and several sites I use are broken and require additional options (cookies) for even basic functionality.
  - uquaiH9c 6y ago
    Which sounds perfectly normal for a webscrapper. Are those issues you describe fixed in not merged PRs in youtube-dl and fixed in merged PR in youtube-dlc?
- Arcorann 6y ago
  I use youtube-dl to download from Nico Nico Douga, which means having to restart the download every few minutes because the program doesn't send heartbeat requests. There's been a PR to fix this for a couple of years now [1], and a more recent updated PR to add encrypted video support [2], both of which are still open. Meanwhile, the corresponding PR on youtube-dlc [3] is actively being looked at. [1] https://github.com/ytdl-org/youtube-dl/pull/18230 https://github.com/ytdl-org/youtube-dl/pull/18230 [2] https://github.com/ytdl-org/youtube-dl/pull/23824 https://github.com/ytdl-org/youtube-dl/pull/23824 [3] https://github.com/blackjack4494/youtube-dlc/pull/55 https://github.com/blackjack4494/youtube-dlc/pull/55
  - raziel2p 6y ago
    It makes sense that it's easier to give <100 PRs attention than 25k+. If the DLC project grows, you can't know if the same won't happen again. Not saying forks should never happen - streamlink and KeepassXC are examples of great forks off the top of my mind - but you can't judge just on anecdotal stale PRs.
    - masklinn 6y ago
      > It makes sense that it's easier to give <100 PRs attention than 25k+. ydl has 700 open PRs, not 25k+. ydl hasn't had 25k+ PRs in its lifetime either, it's had 4k: GH issues and PRs are the same object, so the sequence is shared.
      - hombre_fatal 6y ago
        Doesn't change the point. All of the fork's 55 PRs except 2 and most of the issues were made by the forker himself as well. It's not like the forker inherited all the open issues/PRs, they just dropped them all. Getting popular and having the masses leave a bunch of issues/PRs is hard. Reviewing code is hard and takes more expertise than writing it. A lot harder than reviewing your own PRs on a motivated (for now) fork.
        yorwba 6y ago
- josteink 6y ago
  > Seems to have been originally spawned from this discussion: "Youtube-dl is dying?" https://github.com/ytdl-org/youtube-dl/issues/26462 https://github.com/ytdl-org/youtube-dl/issues/26462 citing many PRs are not getting merged. As someone who has tried to contribute to YouTube-dl in the past, the absolute disinterest I received and complete lack of follow up on simple bug-fixes caused me to completely dismiss YouTube-dl as a FOSS project worthy of my time. And I say that as someone who sends PRs to projects I’ve visited once, over tiny things like typos or lack of installation/build instructions in the readme. Some of my PRs received a crazy all-or-nothing treatment, asking me to fix huge, long-standing bugs nobody had fixed for years, instead of accepting a 2-line diff which did a provider-specific workaround which solved a real bug, but seemingly not in the “right” way. Completely toxic and hostile. That someone has started a community-fork with a more welcoming and pragmatic approach does not surprise me in the least.
  - pwdisswordfish4 6y ago
    I had basically the same experience. I even got requests for functionality that was already there, as if the maintainers didn't even read or test the patch. There are a number of pull requests of mine that have been waiting open for years, ready to be merged. Maybe I'll have more luck with this fork.
- nonbirithm 6y ago
  There is an issue preventing the download of Watch Later playlists that has been open for over a year. The core maintainer, dstftw, closes all reports related to it with "duplicate" while never giving a reason or the original issue. See this issue I filed for an example. https://github.com/ytdl-org/youtube-dl/issues/23860 https://github.com/ytdl-org/youtube-dl/issues/23860 He closed it without even addressing the issue of him closing the related issues that I explicitly brought up. There are scores of issues describing the same issue, but all of them are closed, and more of them are still being submitted. This has been going on since April of last year. It's making me realize that sometimes a fork really is the only way to get around the problem that you're not in control of the project and there are bugs that many people need to be fixed.
  - tardyp 6y ago
    Closing duplicated issues looks okay to me. The author not interested in fixing this issue himself looks okay to me. Is he refusing fix proposals about it?
    - dmd 6y ago
      Yes; there are now several hundred pending PRs for both fixes and new extractors, all ignored.
DarthGhandi 6y ago
To nitpick: It's not scraping, the youtube.com undocumented api changes all the time, in my tinfoil view probably to mess with things like youtube-dl and 3rd party apps.
- dewey 6y ago
  youtube-dl works for thousands of sites, most of them with web scraping. What are you nitpicking about?
  - DarthGhandi 6y ago
    > What are you nitpicking about? Verbatim from my comment: "the youtube.com undocumented api" Thanks for reading.
    - dewey 6y ago
      The top level comment you are replying to and nitpicked the "scraping" verb doesn't say anything about YouTube or the YouTube API.
      - outadoc 6y ago
        Yes, thanks, I was using "scraping" in a more generic way, since I'm sure it does actually use scraping for most video providers. In fact I'm pretty sure I've seen youtube-dl mention downloading the HTML page when using it with YouTube, so maybe it does even scrape YouTube.
- lozenge 6y ago
  YouTube supports Smart TVs, games consoles and other platforms where updating is slow or nearly impossible. Its API is stable for that reason.
  - DarthGhandi 6y ago
    Why do third party apps that utilize it constantly break then? Please explain.
malikNF 6y ago
Especially considering the last release to youtube-dl was just 6 days ago. Guess OP picked a bad time to post the fork.
- theon144 6y ago
  Still sitting at 725 unmerged and/or unrejected PRs though.
  - ekianjo 6y ago
    Is there anything bad about that number? Something which is very popular is bound to get a lot of PRs, and reviewing them takes time.
  - kzrdude 6y ago
    The job of a maintainer is to reject PRs. So yeah, they are falling behind in saying no, maybe they just don't want to say no, so they let them sit. Happens everywhere.
  - xorcist 6y ago
    Let's hope the fork merges those 700 PRs and see which project is maintained in five years. Keeping software well maintained involves a lot of saying no, unfortunately.
    - chrisseaton 6y ago
      > Keeping software well maintained involves a lot of saying no, unfortunately. Right... and the youtube-dl maintainers aren't doing that.
      - slenk 6y ago
        Wouldn't not merging a PR be the same as saying no
        chrisseaton 6y ago
danielheath 6y ago
Getting a fix merged to youtube-dl is quite an endeavour. No criticism to the fantastic maintainers - I encourage them to run their excellent project however suits them best - but I’m not going to try again in future.
- dylan604 6y ago
  Are PR visible to other users where they could make the same change to their local version even if the maintainer does not accept the PR in a timely manner? I know it is no where near the same, but just wondering.
  - InvaderFizz 6y ago
    Yes, on any PR you can click the source branch and see it. This is useful if a project is straightforward to build. You can just git clone the PR branch and build with that fix.
iforgotpassword 6y ago
This seems indeed a confusing post/fork. youtube-dl is already very successful and popular. I use it almost every day. So what is it about youtube-dlc that makes it special compared to the other 12k forks? You scroll down to the readme and it's the exact same, just that someone replaced -dl with -dlc.
jccalhoun 6y ago
the submitter posted this on reddit: https://www.reddit.com/r/DataHoarder/comments/ir8ic6/youtubedlc_an_active_fork_of_youtubedl/ https://www.reddit.com/r/DataHoarder/comments/ir8ic6/youtube...

capableweb 6y ago

Seems to have been originally spawned from this discussion: "Youtube-dl is dying?" https://github.com/ytdl-org/youtube-dl/issues/26462 https://github.com/ytdl-org/youtube-dl/issues/26462 citing many PRs are not getting merged. Anecdote: youtube-dl still works for all the sites I'm using it for.

BiteCode_dev 6y ago
I maintain several open source projects. Sometimes PR are not merged for years. I'm not payed for this, I'm not even thanked for this most of the time, so I'm certainly not going to set any deadline. But forking might be a solution, if you can give the necessary horse power for the long run. Most can't.
- claudeganon 6y ago
  It’s odd to me how nontechnical people have migrated to GitHub to make bug reports on software they use. The other day, I was trying to use a Python library to scrape some stuff. It kept breaking. I looked on the issues page and there were many, essentially useless bug reports asking them to fix the issue. I opened up the script and changed about 7 lines and it worked fine.
  - BiteCode_dev 6y ago
    Well, they look for a channel to communicate their frustration. Most are closed. This one is open. The water follows the path of least resistance.
  - chucky 6y ago
    There are two factors at play here I think. The first is that Github has become quite popular. The second is that Github has worked a lot on usability - it's now very easy for users who are not very technical to create an account and report issues. At times I imagine there could be some kind of "I know how to report bugs"-certification that people could somehow earn. Projects could then use this as a barrier to prevent getting too many useless issue reports. The problem really goes both ways - as a software dev I'm disincentivized to report detailed bugs to projects, because there's a high risk it will get drowned out in the noise. Some projects have quite specific templates for reporting issues. This helps a bit, but you still need someone to go in and remove issues that don't match the template, and few people want to volunteer for that work.
    - claudeganon 6y ago
      Yeah, I guess it’s just an entirely different kind of engagement. I mostly use GitHub to find off the shelf solutions to whatever problem I’m having or to avoid doing the work myself. If I run into any problems I have with something from there, I either try and fix it by tweaking the repository, using another, or just building the thing myself. I think the last time I reported an issue on an external repository was because of something I was made to use at work.
  - OzzyB 6y ago
    I've noticed this trend too -- but I've also noticed people starting GitHub Repos for websites/apps just for the Issues. No code, no PRs, just a way for the public to create Issues. Not sure what this means for the "culture of code" other than things are blurring...
    - claudeganon 6y ago
      I think a lot of it is maybe bleed over from gaming communities, especially emulators, maybe rom hacks too? I’ve noticed barrages of useless bug reports on those repositories. But I’ve been surprised to find them on far afield stuff as well.
    - theidel 6y ago
      At least there is a feature called 'discussions' coming soon to github. That should hopefully minimize the abuse of issues. But that is also due to the fact of current github. Not everyone is displaying their email and there is no other communication channel.
dependenttypes 6y ago
I find it gross that dstftw (the core youtubedl maintainer) censored the part about youtube-dlc from that issue.
- theon144 6y ago
  CTRL+F'ing `youtube-dlc` comes up with several hits - was there more that is missing now?
  - dependenttypes 6y ago
    Check the edit history of the first post.
  - zo1 6y ago
    You can look at the edit history of that comment. You'll see that dstftw edited it to remove references to the youtube_dlc repo. https://github.com/ytdl-org/youtube-dl/issues/26462#issue-688126722 https://github.com/ytdl-org/youtube-dl/issues/26462#issue-68... Unfortunately, can't find a way to link to the edit in question directly. You'll have to click once there to see the diff.
    - saagarjha 6y ago
      Being able to edit people’s comments on GitHub is nice as a maintainer but removing information and completely changing people’s comments is really nasty. Any such effort should be clearly justified, preferably with a visual “edit message” that goes along with the smaller “edited” text.
      - u801e 6y ago
        I don't think that editing comments that you didn't make should ever be a feature. This is something that could easily be abused to make it appear that someone else said something they didn't actually way.
        6y ago
theon144 6y ago
Counter-anecdote: Even YouTube support is somewhat flaky (age-restricted videos, private (as in, locked behind auth) videos, random failures), and several sites I use are broken and require additional options (cookies) for even basic functionality.
- uquaiH9c 6y ago
  Which sounds perfectly normal for a webscrapper. Are those issues you describe fixed in not merged PRs in youtube-dl and fixed in merged PR in youtube-dlc?
Arcorann 6y ago
I use youtube-dl to download from Nico Nico Douga, which means having to restart the download every few minutes because the program doesn't send heartbeat requests. There's been a PR to fix this for a couple of years now [1], and a more recent updated PR to add encrypted video support [2], both of which are still open. Meanwhile, the corresponding PR on youtube-dlc [3] is actively being looked at. [1] https://github.com/ytdl-org/youtube-dl/pull/18230 https://github.com/ytdl-org/youtube-dl/pull/18230 [2] https://github.com/ytdl-org/youtube-dl/pull/23824 https://github.com/ytdl-org/youtube-dl/pull/23824 [3] https://github.com/blackjack4494/youtube-dlc/pull/55 https://github.com/blackjack4494/youtube-dlc/pull/55
- raziel2p 6y ago
  It makes sense that it's easier to give <100 PRs attention than 25k+. If the DLC project grows, you can't know if the same won't happen again. Not saying forks should never happen - streamlink and KeepassXC are examples of great forks off the top of my mind - but you can't judge just on anecdotal stale PRs.
  - masklinn 6y ago
    > It makes sense that it's easier to give <100 PRs attention than 25k+. ydl has 700 open PRs, not 25k+. ydl hasn't had 25k+ PRs in its lifetime either, it's had 4k: GH issues and PRs are the same object, so the sequence is shared.
    - hombre_fatal 6y ago
      Doesn't change the point. All of the fork's 55 PRs except 2 and most of the issues were made by the forker himself as well. It's not like the forker inherited all the open issues/PRs, they just dropped them all. Getting popular and having the masses leave a bunch of issues/PRs is hard. Reviewing code is hard and takes more expertise than writing it. A lot harder than reviewing your own PRs on a motivated (for now) fork.
      - yorwba 6y ago
josteink 6y ago
> Seems to have been originally spawned from this discussion: "Youtube-dl is dying?" https://github.com/ytdl-org/youtube-dl/issues/26462 https://github.com/ytdl-org/youtube-dl/issues/26462 citing many PRs are not getting merged. As someone who has tried to contribute to YouTube-dl in the past, the absolute disinterest I received and complete lack of follow up on simple bug-fixes caused me to completely dismiss YouTube-dl as a FOSS project worthy of my time. And I say that as someone who sends PRs to projects I’ve visited once, over tiny things like typos or lack of installation/build instructions in the readme. Some of my PRs received a crazy all-or-nothing treatment, asking me to fix huge, long-standing bugs nobody had fixed for years, instead of accepting a 2-line diff which did a provider-specific workaround which solved a real bug, but seemingly not in the “right” way. Completely toxic and hostile. That someone has started a community-fork with a more welcoming and pragmatic approach does not surprise me in the least.
- pwdisswordfish4 6y ago
  I had basically the same experience. I even got requests for functionality that was already there, as if the maintainers didn't even read or test the patch. There are a number of pull requests of mine that have been waiting open for years, ready to be merged. Maybe I'll have more luck with this fork.
nonbirithm 6y ago
There is an issue preventing the download of Watch Later playlists that has been open for over a year. The core maintainer, dstftw, closes all reports related to it with "duplicate" while never giving a reason or the original issue. See this issue I filed for an example. https://github.com/ytdl-org/youtube-dl/issues/23860 https://github.com/ytdl-org/youtube-dl/issues/23860 He closed it without even addressing the issue of him closing the related issues that I explicitly brought up. There are scores of issues describing the same issue, but all of them are closed, and more of them are still being submitted. This has been going on since April of last year. It's making me realize that sometimes a fork really is the only way to get around the problem that you're not in control of the project and there are bugs that many people need to be fixed.
- tardyp 6y ago
  Closing duplicated issues looks okay to me. The author not interested in fixing this issue himself looks okay to me. Is he refusing fix proposals about it?
  - dmd 6y ago
    Yes; there are now several hundred pending PRs for both fixes and new extractors, all ignored.

DarthGhandi 6y ago

To nitpick: It's not scraping, the youtube.com undocumented api changes all the time, in my tinfoil view probably to mess with things like youtube-dl and 3rd party apps.

dewey 6y ago
youtube-dl works for thousands of sites, most of them with web scraping. What are you nitpicking about?
- DarthGhandi 6y ago
  > What are you nitpicking about? Verbatim from my comment: "the youtube.com undocumented api" Thanks for reading.
  - dewey 6y ago
    The top level comment you are replying to and nitpicked the "scraping" verb doesn't say anything about YouTube or the YouTube API.
    - outadoc 6y ago
      Yes, thanks, I was using "scraping" in a more generic way, since I'm sure it does actually use scraping for most video providers. In fact I'm pretty sure I've seen youtube-dl mention downloading the HTML page when using it with YouTube, so maybe it does even scrape YouTube.
lozenge 6y ago
YouTube supports Smart TVs, games consoles and other platforms where updating is slow or nearly impossible. Its API is stable for that reason.
- DarthGhandi 6y ago
  Why do third party apps that utilize it constantly break then? Please explain.

malikNF 6y ago

Especially considering the last release to youtube-dl was just 6 days ago. Guess OP picked a bad time to post the fork.

theon144 6y ago
Still sitting at 725 unmerged and/or unrejected PRs though.
- ekianjo 6y ago
  Is there anything bad about that number? Something which is very popular is bound to get a lot of PRs, and reviewing them takes time.
- kzrdude 6y ago
  The job of a maintainer is to reject PRs. So yeah, they are falling behind in saying no, maybe they just don't want to say no, so they let them sit. Happens everywhere.
- xorcist 6y ago
  Let's hope the fork merges those 700 PRs and see which project is maintained in five years. Keeping software well maintained involves a lot of saying no, unfortunately.
  - chrisseaton 6y ago
    > Keeping software well maintained involves a lot of saying no, unfortunately. Right... and the youtube-dl maintainers aren't doing that.
    - slenk 6y ago
      Wouldn't not merging a PR be the same as saying no
      - chrisseaton 6y ago

danielheath 6y ago

Getting a fix merged to youtube-dl is quite an endeavour. No criticism to the fantastic maintainers - I encourage them to run their excellent project however suits them best - but I’m not going to try again in future.

dylan604 6y ago
Are PR visible to other users where they could make the same change to their local version even if the maintainer does not accept the PR in a timely manner? I know it is no where near the same, but just wondering.
- InvaderFizz 6y ago
  Yes, on any PR you can click the source branch and see it. This is useful if a project is straightforward to build. You can just git clone the PR branch and build with that fix.

iforgotpassword 6y ago

This seems indeed a confusing post/fork. youtube-dl is already very successful and popular. I use it almost every day. So what is it about youtube-dlc that makes it special compared to the other 12k forks? You scroll down to the readme and it's the exact same, just that someone replaced -dl with -dlc.

jccalhoun 6y ago

the submitter posted this on reddit: https://www.reddit.com/r/DataHoarder/comments/ir8ic6/youtubedlc_an_active_fork_of_youtubedl/ https://www.reddit.com/r/DataHoarder/comments/ir8ic6/youtube...