10 ms·

You-get: Dumb downloader that scrapes the web

397 points by Anon84 2y ago

deleted 2y ago
[deleted]
politelemon 2y ago
It seems they do not want you to report an issue without an accompanying fix for it. > If you would like to report a problem you find when using you-get, please open a Pull Request, which should include [snip] Can't say I've encountered this before.
- kylecazar 2y ago
  They want you to just submit a PR with a test that, if passed, would indicate the problem for you is fixed.
  - thangngoc89 2y ago
    What happens if you don’t know Python? Python is a relatively easy language to learn but no way I’m gonna learn Python just to report an issue
    - js8 2y ago
      The same thing that happens if the author of the OSS you use doesn't know English.
    - nunez 2y ago
      That's exactly it. They put up a gate that blocks low-effort issues that only add busywork. I like it!
    - Filligree 2y ago
      Good chance you wouldn't be writing good bug reports either, then. Github issues have enough noise that a first-pass filter like this feels like a good idea, even if it has some false positives.
      - papichulo2023 2y ago
        I fail to see the logic in your comment. Just another case of Goodhart's law.
        deleted 2y ago
        [deleted]
        achierius 2y ago
        This isn't really a metric though. It's a formal existence proof that the bug exists. The key difference IMO is that you have to create a test which A) looks (to the maintainer) like it should pass, while simultaneously B) not passing. It's much harder to game. There are other cases where Goodharts Law fails as well: consider quant firms, where the "metric" used to judge a trader is basically how much money you pull in. Seems to be working fine for them
        dartos 2y ago
        If you can’t describe your bug in a test, then you probably can’t describe it sufficiently in English either. Seems to make sense
      - latexr 2y ago
        This in no way aligns with reality. I frequently interact with users who can’t code at all but make good bug reports. One of the best ways to ensure success is to have a form (GitHub allows creating those) which describe exactly what is necessary and guide people in the right direction. What you're saying is even worse, since you’re implying someone could be an expert computer programmer or power user, but because they’re unfamiliar with the specific language this project chose, they are incapable of making good bug reports. That makes no sense.
    - dartos 2y ago
      Then you don’t get to contribute bug reports. Perfectly fine rule for a maintainer to have.
    - dotancohen 2y ago
      If the bug is egregious enough, somebody else will find it. If the bug is important enough to you but esoteric, then ask on a forum or enlist the help of someone you know who does know Python. How do you currently submit bug reports on e.g. MS Word or Adobe Photoshop? This way is certainly more open than those commonly-deployed software.
    - epcoa 2y ago
      Did you (or anyone) in this thread look to see exactly what they are looking for with their provided examples? https://github.com/soimort/you-get/pull/2680/commits/313b8d28d55611be78ebcabce14e11301a65349e https://github.com/soimort/you-get/pull/2680/commits/313b8d2... You do not need to know Python deeply to construct what they are expecting. They’re not actually looking for a unit test or something.
      - latexr 2y ago
        > Did you (or anyone) in this thread look to see exactly what they are looking for with their provided examples? I did. And I looked at all examples of “good commits”, not just the trivial ones. https://github.com/soimort/you-get/pull/2685/files https://github.com/soimort/you-get/pull/2685/files That’s already complex for someone unfamiliar with the software (which might nonetheless be able to open a competent bug report).
        deleted 2y ago
        [deleted]
  - sigseg1v 2y ago
    I kind of like this. It's a more formal proof of concept. You prove the bug exists by writing a failing test. If they cannot construct a failing test then it's either too hard to mock or reproduce (and therefore maybe not even worth fixing, for a free tool), or it's impossible because it's not a bug. Frees up maintainer time from dealing with reports that aren't bugs.
    - latexr 2y ago
      > If they cannot construct a failing test then it's either too hard to mock or reproduce (…), or it's impossible because it's not a bug. Or, you know, the user is not a developer. Or is unfamiliar with Python, or their test suite, or git, or… It is perfectly possible to be good at reporting bugs but be incapable of submitting pull requests.
      - newaccount74 2y ago
        The problem with popular tools is that they have more bugs that can be fixed. So bug reports are pretty much worthless: You know that there are 1000 bugs out there, but you only have resources to fix 10 of them. By asking users to provide reproducible test cases, you can massively reduce the amount of work you have to do. Of course that means 90% of bugs will never be reported. But since you don't have the resources to fix them anyway, why not just focus on the bugs that can be reproduced and come with a test case...
        justaj 2y ago
        I don't think it's necessarily about fixing those bugs, but I think a lot of times it's more about at least having those bugs be documented in order to raise awareness on (probable) issues down the line for whoever would want to use that project in the future.
        latexr 2y ago
        You missed the point entirely. It’s your prerogative if and how you want to limit the amount of people who can contribute, but I was explicitly replying to someone claiming that a person’s inability to code is in any way related to the validity or importance of the bug.
- onionisafruit 2y ago
  Interesting. I like the idea of encouraging people to try creating a test or even a whole fix, but saying that’s all you will accept is a bit much. On the other hand, I’m not doing the work to maintain you-get. I don’t know what they deal with. This may be an effective way to filter a flood of repetitive issues from people who don’t know how to run a command line program.
  - probably_wrong 2y ago
    I believe there are two extremes. On one end you get a bunch of repetitive non-issues, while on the other end you only get issues about (say) bugs in FreeBSD 13.3 because only hard-core users have the skills and patience to follow THE PROCESS. I know how to make an isolated virtual environment, install the package, make a fork, create a test and make a PR. But I don't know whether I care enough about a random project to actually do it.
- wccrawford 2y ago
  As the other commenter said, they want a failing test, not a fix. A detailed description of the encountered problem; At least one commit, addressing the problem through some unit test(s). Examples of good commits: #2675, #2680, #2685 "Addressing" is probably a bad word to use here. "Demonstrating" would have been better, IMO.
  - tylerchilds 2y ago
    the most expensive piece of writing software is scoping work. i’m almost tempted to add a test suite just to give people more agency over my output because right now i’m only soliciting feedback in person to cut down on internet bullshit, like what happened to xz-utils
- thih9 2y ago
  It’s relatively easy to write a failing test and it massively cuts down the work related to moderating issues. Also, reduces the danger of github issues turning into a support forum. If this results in the project being easier to maintain and being maintained longer, then I’m fine with this.
  - seneca 2y ago
    > It’s relatively easy to write a failing test and it massively cuts down the work related to moderating issues. Relative to what? Learning someone else's code base well enough to write a useful test is not trivial. It's not a bad method, but the vast majority of users won't be capable of writing a test that encapsulates their issue.
    - chucksmash 2y ago
      In the case of this tool, adding a failing test case looks trivial if you've got the URL of a page it fails on. Provided the maintainer is willing to provide some minimal guidance to issue reporters who lack the necessary know-how, it even seems like a clever back door way of helping people learn to contribute to open source.
- zufallsheld 2y ago
  Serverspec does the same: https://github.com/mizzy/serverspec?tab=readme-ov-file#maintenance-policy-of-serverspecspecinfra https://github.com/mizzy/serverspec?tab=readme-ov-file#maint...
- omoikane 2y ago
  The Chinese version of the text has an extra header line that translates to "to prevent abuse via GitHub Issues, we are not accepting general issues". An earlier commit has this for the English text: `you-get` is currently experimenting with an aggressive approach to handling issues. Namely, a bug report must be addressed with some code via a pull request. https://github.com/soimort/you-get/commit/75b44b83826b3c2d9a298df1dae3e2805b151205 https://github.com/soimort/you-get/commit/75b44b83826b3c2d9a... Maybe they got too much spam. By the way, `tests/test.py` seems to just run the extractors against various websites directly. I can't find where it's mocking out network requests and replies. Maybe this is to simplify the process for people creating pull requests?
  - godelski 2y ago
    I can get this, but I aggressively report accounts and issues. I'm not sure how GitHub handles them but they seem to not come back. Though what I'm unsure how to deal with is legitimate users being idiotic. For example, recently one issue was opened that asked where the source code was. Not only was there a directory named "src" but there were some links in the readme to specific parts. While I do appreciate GitHub and places like hugging face [0], there are a lot of very aggressive and demanding noobs. I'd like ways to handle them better.... I'm tired of people yelling at me because 5 year old research code no longer works out of the box or because you've never touched code before. [0] check any hugging face issue and you'll see far more spam. Same accounts will open multiple issues that just barate owners and hugging face makes it difficult to report these accounts.
    - throwaway314155 2y ago
      The solution is to ignore them and close their issue. Open source maintainers have enough to worry about and are unpaid, it's okay to be a little dictatorial when it comes to "bad questions".
      - godelski 2y ago
        That's not a solution. It addresses the specific issue but does nothing to prevent future similar issues. A solution to a cold is not handing someone a tissue. I like that these platforms are open to everyone but at the same time there are a lot of people who have no business participating. Being able to filter those people out is unfortunately a necessary tool to not get overloaded. Worse, I find that due to this many open source maintainers and up being quick to close issues and say rtfm. I can't tell you how many times I've had this happen where in my opening issue I quote the fm and even include a reproducible test. It's also common to just close and say "not our problem".
- deleted 2y ago
  [deleted]
KTibow 2y ago
Can someone explain why this is better than yt-dlp
- deleted 2y ago
  [deleted]
- billsunshine 2y ago
  [flagged]
  - Etheryte 2y ago
    Please don't litter HN with LLM generated slop, this is actively reducing the quality of discussion. No one wants a future HN where people just spam LLM responses at one another.
- uniqueuid 2y ago
  That's an interesting question. They only depend on a single library, but I wonder how much code is really their own. I found it curious, for example, that there is a dedicated mp4 joiner (I mean, if you already have ffmpeg, there is probably no way you can do it better yourself). https://github.com/soimort/you-get/blob/develop/src/you_get/processor/join_mp4.py https://github.com/soimort/you-get/blob/develop/src/you_get/...
  - radicality 2y ago
    That is interesting, huh, yeah they list ffmpeg as a dependency so I wonder what that didn’t cover for them. Though there are some cases where using pure ffmpeg is just to difficult or impossible. Recently i had such a case where I wanted to merge multiple video files from my GoPro (they come out in 4GB parts if the video is longer than that), but while keeping / correctly merging all metadata such as date / accelerometer / gps / any custom binary streams. Ended up using this and worked great https://github.com/gyroflow/mp4-merge https://github.com/gyroflow/mp4-merge
- grugagag 2y ago
  How did you infer better than yt-dlp? I think the more the better when it comes to this space as google fights back.
vanjajaja1 2y ago
> Search on Google Videos and download > $ you-get "Richard Stallman eats" I don't often read instruction manuals, but this time I did and I found this gross easter egg
- crowdyriver 2y ago
  oh god
  - vanjajaja1 2y ago
    'dont' was implied, though apparently not inferred
dotancohen 2y ago
Can it back up a text webpage? Can it remove popups for newsletters, or subscription, or logins, or cookies' notifications? Can it read pages that require signing in?
demberto 2y ago
this different from JDownloader2?
tcsenpai 2y ago
I like this. I am imagining a companion extension for chrome/ff that uses you-get as a backend to implement it in a seamless way. Forward thinking idea: imagine going on youtube and have you-get extension bypass the youtube player and playing the content directly without ads. When I say youtube I might also say any other platform.
- mikojan 2y ago
  Sounds like FastStream Video Player https://addons.mozilla.org/en-US/firefox/addon/faststream/?utm_source=addons.mozilla.org&utm_medium=referral&utm_content=search https://addons.mozilla.org/en-US/firefox/addon/faststream/?u...
xg15 2y ago
I wouldn't exactly call a ytdl-style media downloader with a whole library of site-specific extractors and converters "dumb" but still cool that more projects like ytdl exist.
andai 2y ago
For a while I had expensive internet and low bandwidth, but I loved listening to music and lectures on YouTube. At some point I realized that getting only the audio stream would save me 90% in bandwidth costs. [0] youtube-dl (and yt-dlp) has a flag, I believe -G, which gives you the URL(s) for the requested format/quality. I used the command line on my computer and put the link in VLC. On my phone I had this elaborate workaround involving downloading the file to my VPS first over SSH, then downloading it to my phone, until I realized my phone browser can consume the URL directly, so I set up a PHP frontend for `youtube-dl -G -f bestaudio {url}` It's no longer online and I lost the code, but it was like one line of code. I mention this because you-get seems to support the same usecase (via --url / -u), so I wanted to let people know how useful this is! (While it was online I shared it on some forums and got very positive feedback, people used it for audiobooks etc.) [0] Also playing with screen off saves 80% battery life! YouTube knows these facts and that's why they made background playback (which fetches only audio stream) a paid feature...
- 01HNNWZ0MV43FF 2y ago
  I think it's -x to just rip audio now
- TechDebtDevin 2y ago
  Brave Mobile browser allows turning on background video audio thus eliminating the need for YouTube Premium and similar subscriptions.
  - BiteCode_dev 2y ago
    New Pipe X if you need a full client does that as well.
    - marci 2y ago
      I know of NewPipe (Android) but can't find anything about New Pipe X
      - 2y ago
MattDaEskimo 2y ago
Another library released which lies about what it is to circumvent anti-bot security. Let's just not act surprised when tighter attestation comes in effect.
- ajsnigrutin 2y ago
  This library/program solves problems that people have with pages like youtube... too many ads, no way to download videos for offline use (or archive for when they get removed), and better performance with a native player. If I was forced to watch all the ads on youtube, i wouldn't watch videos there at all.
  - MattDaEskimo 2y ago
    I get it. But I don't. For YouTube you can pay for premium. Consuming not only their resources but also their incredible streaming algorithm for free is just a dirty move. Doing this just puts you into the statistic of bad users that incentives companies like Google to push more intrusive DRM. Which at the end of the day makes us all suffer. Good luck finding an alternative by the way.
    - ajsnigrutin 2y ago
      And I consider them bad advertisers and just don't care. We had the internet without ads. Then we had the internet with small banner ads, that didn't track users but were there because of the page content. Then those ads moved from text/image to animated gif, and they became annoying. Then flash was used and it became a security concern. Then the number of ads went up and up, video was introduced, autoplaying, with audio, and at some point most of the internet is unusable without an adblock. It's not something we, the users, wanted, the advertisers (google included) made the internet unusable without an adblock, and that includes 3 25 second ads on a 1m20s video on youtube. If advertisers returned to non-animated banners above/below the video, we wouldn't have to install adblocks everywhere anymore, and people would see the ad without wanting to kill the advertiser for bad intrusive advertising practices. They started it, they went too far, we're just reacting to what they're doing. And as we've seen with other platforms, even paying for premium doesn't mean no-ads. And still an adblock is needed to remove the in-video ads ("this video is sponsored by shadow vpn audible").
krick 2y ago
Given the title and the first few sentences from a description I assumed that it's some heuristic-based tool to try and grab whatever there is on the page, which would be useful if there's no tool which implemented the support for this site (which in most cases just means "yt-dlp doesn't support it"). But apparently it's also extractor-based with a separate extractor for each somewhat-popular source. So, basically it's just less sophisticated clone of yt-dlp?
jdthedisciple 2y ago
Anybody else getting this error constantly? you-get: [error] oops, something went wrong. you-get: don't panic, c'est la vie. please try the following steps: you-get: (1) Rule out any network problem. you-get: (2) Make sure you-get is up-to-date. you-get: (3) Check if the issue is already known, on you-get: https://github.com/soimort/you-get/wiki/Known-Bugs you-get: https://github.com/soimort/you-get/issues you-get: (4) Run the command with '--debug' option, you-get: and report this issue with the full output. Tried with debug flag but didn't really help pattern = str(pattern, 'latin1') ^^^^^^^^^^^^^^^^^^^^^^ TypeError: decoding to str: need a bytes-like object, NoneType found I was curious to see if it can bypass age restriction (though I tried on non-age-restricted video too with the same error).
- tcper 2y ago
  Same error, not work at all.
- dmuth 2y ago
  Check your Python version. Try switching to the newest version of Python with something like Pyenv, or even try spinning up a Docker image with something like: docker run -it -v $(pwd):/mnt python:3.13 bash ...and try using the utility there. Good luck!
natch 2y ago
Is this just a fork of yet-dlp with credits rewritten?
- EasyMark 2y ago
  Why not diff the code bases and find out?
  - natch 2y ago
    Too hard, I was hoping you would do it.
fnoobnar 2y ago
I’m not sure I understand why Bandcamp is on the list of supported sites: they allow you to just download the files on the condition you first pay the artist for them. The fact you can download it with this tool is because the artist is letting you listen to it for free before buying it. Downloading it with this tool seems totally unnecessary and a bit of a jerk move. Bandcamp hosts mostly small and independent artists and labels.
- khaki54 2y ago
  I presume you could subscribe and still use this tool? People use automation tools like this to download things that they already pay for because it saves them the effort of logging into 5 different apps depending on which walled garden it's in.
  - hluska 2y ago
    Do artists get paid on Bandcamp if they bypass the login?
  - fnoobnar 2y ago
    It’s Bandcamp: you buy the album and they give you a link to download it in whatever format you want so you can listen to it using whatever app or device you wish. You can come back later and download it as many times as you like.
    - Gormo 2y ago
      What if I don't want to manually follow links or "come back later", but just add the album I purchased to the config for the same script I use for the rest of my media library?
- lovethevoid 2y ago
  Their list of supported sites isn't a declaration of where you should use this tool for moralistic reasons. It's just a list of popular sites it works on.
- 2y ago
wanderingmind 2y ago
Nice work. But as a consumer, Why should I use you-get over yt-dlp? What are its strengths over yt-dlp, which works quiet well on a huge range of websites[1] [1] https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites....
jdasdf 2y ago
How you get over the login barriers for youtube?
rldjbpin 2y ago
the project appears to have an inaccurate description. perhaps it is the long-term goal, but today it behaves more or less like a subset of what yt-dlp does. there is some polish to the interface, so it might still be worth trying out if it does what you need it for.

397 points by Anon84 2y ago

deleted 2y ago

[deleted]

It seems they do not want you to report an issue without an accompanying fix for it. > If you would like to report a problem you find when using you-get, please open a Pull Request, which should include [snip] Can't say I've encountered this before.

kylecazar 2y ago
They want you to just submit a PR with a test that, if passed, would indicate the problem for you is fixed.
- thangngoc89 2y ago
  What happens if you don’t know Python? Python is a relatively easy language to learn but no way I’m gonna learn Python just to report an issue
  - js8 2y ago
    The same thing that happens if the author of the OSS you use doesn't know English.
  - nunez 2y ago
    That's exactly it. They put up a gate that blocks low-effort issues that only add busywork. I like it!
  - Filligree 2y ago
    Good chance you wouldn't be writing good bug reports either, then. Github issues have enough noise that a first-pass filter like this feels like a good idea, even if it has some false positives.
    - papichulo2023 2y ago
      I fail to see the logic in your comment. Just another case of Goodhart's law.
      - deleted 2y ago
        [deleted]
      - achierius 2y ago
        This isn't really a metric though. It's a formal existence proof that the bug exists. The key difference IMO is that you have to create a test which A) looks (to the maintainer) like it should pass, while simultaneously B) not passing. It's much harder to game. There are other cases where Goodharts Law fails as well: consider quant firms, where the "metric" used to judge a trader is basically how much money you pull in. Seems to be working fine for them
      - dartos 2y ago
        If you can’t describe your bug in a test, then you probably can’t describe it sufficiently in English either. Seems to make sense
    - latexr 2y ago
      This in no way aligns with reality. I frequently interact with users who can’t code at all but make good bug reports. One of the best ways to ensure success is to have a form (GitHub allows creating those) which describe exactly what is necessary and guide people in the right direction. What you're saying is even worse, since you’re implying someone could be an expert computer programmer or power user, but because they’re unfamiliar with the specific language this project chose, they are incapable of making good bug reports. That makes no sense.
  - dartos 2y ago
    Then you don’t get to contribute bug reports. Perfectly fine rule for a maintainer to have.
  - dotancohen 2y ago
    If the bug is egregious enough, somebody else will find it. If the bug is important enough to you but esoteric, then ask on a forum or enlist the help of someone you know who does know Python. How do you currently submit bug reports on e.g. MS Word or Adobe Photoshop? This way is certainly more open than those commonly-deployed software.
  - epcoa 2y ago
    Did you (or anyone) in this thread look to see exactly what they are looking for with their provided examples? https://github.com/soimort/you-get/pull/2680/commits/313b8d28d55611be78ebcabce14e11301a65349e https://github.com/soimort/you-get/pull/2680/commits/313b8d2... You do not need to know Python deeply to construct what they are expecting. They’re not actually looking for a unit test or something.
    - latexr 2y ago
      > Did you (or anyone) in this thread look to see exactly what they are looking for with their provided examples? I did. And I looked at all examples of “good commits”, not just the trivial ones. https://github.com/soimort/you-get/pull/2685/files https://github.com/soimort/you-get/pull/2685/files That’s already complex for someone unfamiliar with the software (which might nonetheless be able to open a competent bug report).
      - deleted 2y ago
        [deleted]
- sigseg1v 2y ago
  I kind of like this. It's a more formal proof of concept. You prove the bug exists by writing a failing test. If they cannot construct a failing test then it's either too hard to mock or reproduce (and therefore maybe not even worth fixing, for a free tool), or it's impossible because it's not a bug. Frees up maintainer time from dealing with reports that aren't bugs.
  - latexr 2y ago
    > If they cannot construct a failing test then it's either too hard to mock or reproduce (…), or it's impossible because it's not a bug. Or, you know, the user is not a developer. Or is unfamiliar with Python, or their test suite, or git, or… It is perfectly possible to be good at reporting bugs but be incapable of submitting pull requests.
    - newaccount74 2y ago
      The problem with popular tools is that they have more bugs that can be fixed. So bug reports are pretty much worthless: You know that there are 1000 bugs out there, but you only have resources to fix 10 of them. By asking users to provide reproducible test cases, you can massively reduce the amount of work you have to do. Of course that means 90% of bugs will never be reported. But since you don't have the resources to fix them anyway, why not just focus on the bugs that can be reproduced and come with a test case...
      - justaj 2y ago
        I don't think it's necessarily about fixing those bugs, but I think a lot of times it's more about at least having those bugs be documented in order to raise awareness on (probable) issues down the line for whoever would want to use that project in the future.
      - latexr 2y ago
        You missed the point entirely. It’s your prerogative if and how you want to limit the amount of people who can contribute, but I was explicitly replying to someone claiming that a person’s inability to code is in any way related to the validity or importance of the bug.
onionisafruit 2y ago
Interesting. I like the idea of encouraging people to try creating a test or even a whole fix, but saying that’s all you will accept is a bit much. On the other hand, I’m not doing the work to maintain you-get. I don’t know what they deal with. This may be an effective way to filter a flood of repetitive issues from people who don’t know how to run a command line program.
- probably_wrong 2y ago
  I believe there are two extremes. On one end you get a bunch of repetitive non-issues, while on the other end you only get issues about (say) bugs in FreeBSD 13.3 because only hard-core users have the skills and patience to follow THE PROCESS. I know how to make an isolated virtual environment, install the package, make a fork, create a test and make a PR. But I don't know whether I care enough about a random project to actually do it.
wccrawford 2y ago
As the other commenter said, they want a failing test, not a fix. A detailed description of the encountered problem; At least one commit, addressing the problem through some unit test(s). Examples of good commits: #2675, #2680, #2685 "Addressing" is probably a bad word to use here. "Demonstrating" would have been better, IMO.
- tylerchilds 2y ago
  the most expensive piece of writing software is scoping work. i’m almost tempted to add a test suite just to give people more agency over my output because right now i’m only soliciting feedback in person to cut down on internet bullshit, like what happened to xz-utils
thih9 2y ago
It’s relatively easy to write a failing test and it massively cuts down the work related to moderating issues. Also, reduces the danger of github issues turning into a support forum. If this results in the project being easier to maintain and being maintained longer, then I’m fine with this.
- seneca 2y ago
  > It’s relatively easy to write a failing test and it massively cuts down the work related to moderating issues. Relative to what? Learning someone else's code base well enough to write a useful test is not trivial. It's not a bad method, but the vast majority of users won't be capable of writing a test that encapsulates their issue.
  - chucksmash 2y ago
    In the case of this tool, adding a failing test case looks trivial if you've got the URL of a page it fails on. Provided the maintainer is willing to provide some minimal guidance to issue reporters who lack the necessary know-how, it even seems like a clever back door way of helping people learn to contribute to open source.
zufallsheld 2y ago
Serverspec does the same: https://github.com/mizzy/serverspec?tab=readme-ov-file#maintenance-policy-of-serverspecspecinfra https://github.com/mizzy/serverspec?tab=readme-ov-file#maint...
omoikane 2y ago
The Chinese version of the text has an extra header line that translates to "to prevent abuse via GitHub Issues, we are not accepting general issues". An earlier commit has this for the English text: `you-get` is currently experimenting with an aggressive approach to handling issues. Namely, a bug report must be addressed with some code via a pull request. https://github.com/soimort/you-get/commit/75b44b83826b3c2d9a298df1dae3e2805b151205 https://github.com/soimort/you-get/commit/75b44b83826b3c2d9a... Maybe they got too much spam. By the way, `tests/test.py` seems to just run the extractors against various websites directly. I can't find where it's mocking out network requests and replies. Maybe this is to simplify the process for people creating pull requests?
- godelski 2y ago
  I can get this, but I aggressively report accounts and issues. I'm not sure how GitHub handles them but they seem to not come back. Though what I'm unsure how to deal with is legitimate users being idiotic. For example, recently one issue was opened that asked where the source code was. Not only was there a directory named "src" but there were some links in the readme to specific parts. While I do appreciate GitHub and places like hugging face [0], there are a lot of very aggressive and demanding noobs. I'd like ways to handle them better.... I'm tired of people yelling at me because 5 year old research code no longer works out of the box or because you've never touched code before. [0] check any hugging face issue and you'll see far more spam. Same accounts will open multiple issues that just barate owners and hugging face makes it difficult to report these accounts.
  - throwaway314155 2y ago
    The solution is to ignore them and close their issue. Open source maintainers have enough to worry about and are unpaid, it's okay to be a little dictatorial when it comes to "bad questions".
    - godelski 2y ago
      That's not a solution. It addresses the specific issue but does nothing to prevent future similar issues. A solution to a cold is not handing someone a tissue. I like that these platforms are open to everyone but at the same time there are a lot of people who have no business participating. Being able to filter those people out is unfortunately a necessary tool to not get overloaded. Worse, I find that due to this many open source maintainers and up being quick to close issues and say rtfm. I can't tell you how many times I've had this happen where in my opening issue I quote the fm and even include a reproducible test. It's also common to just close and say "not our problem".
deleted 2y ago
[deleted]

KTibow 2y ago

Can someone explain why this is better than yt-dlp

deleted 2y ago
[deleted]
billsunshine 2y ago
[flagged]
- Etheryte 2y ago
  Please don't litter HN with LLM generated slop, this is actively reducing the quality of discussion. No one wants a future HN where people just spam LLM responses at one another.
uniqueuid 2y ago
That's an interesting question. They only depend on a single library, but I wonder how much code is really their own. I found it curious, for example, that there is a dedicated mp4 joiner (I mean, if you already have ffmpeg, there is probably no way you can do it better yourself). https://github.com/soimort/you-get/blob/develop/src/you_get/processor/join_mp4.py https://github.com/soimort/you-get/blob/develop/src/you_get/...
- radicality 2y ago
  That is interesting, huh, yeah they list ffmpeg as a dependency so I wonder what that didn’t cover for them. Though there are some cases where using pure ffmpeg is just to difficult or impossible. Recently i had such a case where I wanted to merge multiple video files from my GoPro (they come out in 4GB parts if the video is longer than that), but while keeping / correctly merging all metadata such as date / accelerometer / gps / any custom binary streams. Ended up using this and worked great https://github.com/gyroflow/mp4-merge https://github.com/gyroflow/mp4-merge
grugagag 2y ago
How did you infer better than yt-dlp? I think the more the better when it comes to this space as google fights back.

vanjajaja1 2y ago

> Search on Google Videos and download > $ you-get "Richard Stallman eats" I don't often read instruction manuals, but this time I did and I found this gross easter egg

crowdyriver 2y ago
oh god
- vanjajaja1 2y ago
  'dont' was implied, though apparently not inferred

dotancohen 2y ago

Can it back up a text webpage? Can it remove popups for newsletters, or subscription, or logins, or cookies' notifications? Can it read pages that require signing in?

demberto 2y ago

this different from JDownloader2?

tcsenpai 2y ago

I like this. I am imagining a companion extension for chrome/ff that uses you-get as a backend to implement it in a seamless way. Forward thinking idea: imagine going on youtube and have you-get extension bypass the youtube player and playing the content directly without ads. When I say youtube I might also say any other platform.

mikojan 2y ago
Sounds like FastStream Video Player https://addons.mozilla.org/en-US/firefox/addon/faststream/?utm_source=addons.mozilla.org&utm_medium=referral&utm_content=search https://addons.mozilla.org/en-US/firefox/addon/faststream/?u...

xg15 2y ago

I wouldn't exactly call a ytdl-style media downloader with a whole library of site-specific extractors and converters "dumb" but still cool that more projects like ytdl exist.

andai 2y ago

For a while I had expensive internet and low bandwidth, but I loved listening to music and lectures on YouTube. At some point I realized that getting only the audio stream would save me 90% in bandwidth costs. [0] youtube-dl (and yt-dlp) has a flag, I believe -G, which gives you the URL(s) for the requested format/quality. I used the command line on my computer and put the link in VLC. On my phone I had this elaborate workaround involving downloading the file to my VPS first over SSH, then downloading it to my phone, until I realized my phone browser can consume the URL directly, so I set up a PHP frontend for `youtube-dl -G -f bestaudio {url}` It's no longer online and I lost the code, but it was like one line of code. I mention this because you-get seems to support the same usecase (via --url / -u), so I wanted to let people know how useful this is! (While it was online I shared it on some forums and got very positive feedback, people used it for audiobooks etc.) [0] Also playing with screen off saves 80% battery life! YouTube knows these facts and that's why they made background playback (which fetches only audio stream) a paid feature...

01HNNWZ0MV43FF 2y ago
I think it's -x to just rip audio now
TechDebtDevin 2y ago
Brave Mobile browser allows turning on background video audio thus eliminating the need for YouTube Premium and similar subscriptions.
- BiteCode_dev 2y ago
  New Pipe X if you need a full client does that as well.
  - marci 2y ago
    I know of NewPipe (Android) but can't find anything about New Pipe X
    - 2y ago

MattDaEskimo 2y ago

Another library released which lies about what it is to circumvent anti-bot security. Let's just not act surprised when tighter attestation comes in effect.

ajsnigrutin 2y ago
This library/program solves problems that people have with pages like youtube... too many ads, no way to download videos for offline use (or archive for when they get removed), and better performance with a native player. If I was forced to watch all the ads on youtube, i wouldn't watch videos there at all.
- MattDaEskimo 2y ago
  I get it. But I don't. For YouTube you can pay for premium. Consuming not only their resources but also their incredible streaming algorithm for free is just a dirty move. Doing this just puts you into the statistic of bad users that incentives companies like Google to push more intrusive DRM. Which at the end of the day makes us all suffer. Good luck finding an alternative by the way.
  - ajsnigrutin 2y ago
    And I consider them bad advertisers and just don't care. We had the internet without ads. Then we had the internet with small banner ads, that didn't track users but were there because of the page content. Then those ads moved from text/image to animated gif, and they became annoying. Then flash was used and it became a security concern. Then the number of ads went up and up, video was introduced, autoplaying, with audio, and at some point most of the internet is unusable without an adblock. It's not something we, the users, wanted, the advertisers (google included) made the internet unusable without an adblock, and that includes 3 25 second ads on a 1m20s video on youtube. If advertisers returned to non-animated banners above/below the video, we wouldn't have to install adblocks everywhere anymore, and people would see the ad without wanting to kill the advertiser for bad intrusive advertising practices. They started it, they went too far, we're just reacting to what they're doing. And as we've seen with other platforms, even paying for premium doesn't mean no-ads. And still an adblock is needed to remove the in-video ads ("this video is sponsored by shadow vpn audible").

krick 2y ago

Given the title and the first few sentences from a description I assumed that it's some heuristic-based tool to try and grab whatever there is on the page, which would be useful if there's no tool which implemented the support for this site (which in most cases just means "yt-dlp doesn't support it"). But apparently it's also extractor-based with a separate extractor for each somewhat-popular source. So, basically it's just less sophisticated clone of yt-dlp?

jdthedisciple 2y ago

Anybody else getting this error constantly? you-get: [error] oops, something went wrong. you-get: don't panic, c'est la vie. please try the following steps: you-get: (1) Rule out any network problem. you-get: (2) Make sure you-get is up-to-date. you-get: (3) Check if the issue is already known, on you-get: https://github.com/soimort/you-get/wiki/Known-Bugs you-get: https://github.com/soimort/you-get/issues you-get: (4) Run the command with '--debug' option, you-get: and report this issue with the full output. Tried with debug flag but didn't really help pattern = str(pattern, 'latin1') ^^^^^^^^^^^^^^^^^^^^^^ TypeError: decoding to str: need a bytes-like object, NoneType found I was curious to see if it can bypass age restriction (though I tried on non-age-restricted video too with the same error).

tcper 2y ago
Same error, not work at all.
dmuth 2y ago
Check your Python version. Try switching to the newest version of Python with something like Pyenv, or even try spinning up a Docker image with something like: docker run -it -v $(pwd):/mnt python:3.13 bash ...and try using the utility there. Good luck!

natch 2y ago

Is this just a fork of yet-dlp with credits rewritten?

EasyMark 2y ago
Why not diff the code bases and find out?
- natch 2y ago
  Too hard, I was hoping you would do it.

fnoobnar 2y ago

I’m not sure I understand why Bandcamp is on the list of supported sites: they allow you to just download the files on the condition you first pay the artist for them. The fact you can download it with this tool is because the artist is letting you listen to it for free before buying it. Downloading it with this tool seems totally unnecessary and a bit of a jerk move. Bandcamp hosts mostly small and independent artists and labels.

khaki54 2y ago
I presume you could subscribe and still use this tool? People use automation tools like this to download things that they already pay for because it saves them the effort of logging into 5 different apps depending on which walled garden it's in.
- hluska 2y ago
  Do artists get paid on Bandcamp if they bypass the login?
- fnoobnar 2y ago
  It’s Bandcamp: you buy the album and they give you a link to download it in whatever format you want so you can listen to it using whatever app or device you wish. You can come back later and download it as many times as you like.
  - Gormo 2y ago
    What if I don't want to manually follow links or "come back later", but just add the album I purchased to the config for the same script I use for the rest of my media library?
lovethevoid 2y ago
Their list of supported sites isn't a declaration of where you should use this tool for moralistic reasons. It's just a list of popular sites it works on.
2y ago

wanderingmind 2y ago

Nice work. But as a consumer, Why should I use you-get over yt-dlp? What are its strengths over yt-dlp, which works quiet well on a huge range of websites[1] [1] https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites....

jdasdf 2y ago

How you get over the login barriers for youtube?

rldjbpin 2y ago

the project appears to have an inaccurate description. perhaps it is the long-term goal, but today it behaves more or less like a subset of what yt-dlp does. there is some polish to the interface, so it might still be worth trying out if it does what you need it for.