6 ms·

So glad I exist in a timeline where youtube-dl exists. Having culture locked up in these corporate sites that can wipe it out or remove it in less than a secon

So glad I exist in a timeline where youtube-dl exists. Having culture locked up in these corporate sites that can wipe it out or remove it in less than a second makes me so anxious.

api 9y ago
I am a confirmed packrat. When I hear a really interesting podcast, obscure piece of music, etc. I mirror it to local storage... especially if it's political. Storage is cheap. Doesn't even have to be 'verboten' political. It would be easy for an AI bot trained to curtail e.g. 'hate speech' to delete loads of things that are just unorthodox.
Dirlewanger 9y ago
Agreed, but it also makes me a little sad that we could potentially only have some shitty compressed transcode of a transcode copy of some rare album, since YouTube is plagued with those kinds of uploads. Really makes the loss of what.cd hurt even more.
- isatty 9y ago
  red isn't doing too badly though
  - ams6110 9y ago
    I'm a YouTube Red subscriber, and I also use youtube-dl to allow for offline listening on various devices. Since I'm paying Youtube, and not redistributing my downloads, I don't feel this is any different from DVR-ing a TV show or the old (and legal) practice of copying a vinyl LP to cassette tape so I could listen to it in my car.
    - baldfat 9y ago
      Except the content makers aren't getting revenue since YouTube Red pays for engagement. I LOVE YouTube Red I only use youtube-dl for special use cases.
    - gue5t 9y ago
      It seems likely that the gp is talking about what.cd successor redacted.ch, not the "YouTube Red" paid streaming service.
  - lfam 9y ago
    Red is great, but if we want to preserve our musical heritage, we can't need to hide as we do it. I think that secret libraries are not as resilient as public archives.
- intoverflow2 9y ago
  Only enthusiasts can truly be trusted with archiving culture. Compare what.cd to iTunes. Compare Karagarga to Netflix, people actually go out of their way to create the best version of a movie, maybe the western release has incorrect hardcoded subs and awful picture quality. Someone goes out of their way to find the best version of it which may be something obscure like a Japanese laserdisc, rip it, fix subtitles and audio etc and release it for no monetary gain just the pure love of the media. People actually being paid to release this stuff care far less about it than enthusiasts.
hanklazard 9y ago
Agree completely. I am late to the party as I only heard about this tool a few days ago (from HN, of course), but I'm already archiving some of my favorite obscure videos. Props to the developers.
shortoncash 9y ago
I feel like things will change in the next few years. One of these decentralized video or storage services might take off and change things forever. Perhaps it's wishful thinking, but ever since blockchain technology got rapid adoption (at least among technical people), I can't help but think we're a few user interface innovations on top of decentralized technology away from really seeing a completely different kind of internet.
- unlmtd1 9y ago
  It's already there. IPFS. Just need a way to index it. There's Ethereum. Tezos could potentially do it cheaper.
  - TeMPOraL 9y ago
    And a way to ensure you don't get sued out of your livelihood for the copyrighted file IPFS happened to cache on your machine.
- azeirah 9y ago
  I wonder if you could take a middle step, and host a community-run server for youtube-dl itself; run YouTube-mp3 on an anonymous and decentralized server. Perhaps Ethereum is the right technology for this. Would something like that be possible with Ethereum or otherwise blockchains in general?
  - pavel_lishin 9y ago
    I don't understand blockchain technology in depth, but my understanding is that writing to the blockchain is usually not particularly cheap or fast, especially when writing large amounts of data. A blockchain that fills up with terabytes of pirated movies or mp3s would become difficult to operate, wouldn't it?
    - saalweachter 9y ago
      You'd be using some sort of filecoin-like storage system, where you pay people for distributed storage. If I've been following the whole Ethereum thing correctly, as an outsider who kind of looks down on it, this is right up it's alley. You write some sort of smart contract which allows for the uploading and downloading of music blobs, charges for the service, and if insufficient coins are garnered for storage deletes the least used files. I don't think you need a server or any such thing, and the only way to force a take down would be to take down the underlying services.
ocdtrekkie 9y ago
Out of curiosity: Is youtube-dl something that can be prevented from working? Does a download from youtube-dl look different to a Google server than a legitimate view? I guess it's kinda surprising to me that it still works after all this time, that there isn't some sort of authentication involved to prevent it.
- Mindwipe 9y ago
  YouTube has broken youtube-dl multiple times (a lot). It's a cat and mouse situation. Ultimately YouTube could start properly DRM-ing all their streams. If you watch a film you've bought from Google Play via YouTube it is protected by Widevine DRM, and youtube-dl fails. It seems like YouTube feel it's cheaper at the moment to try and frustrate downloading this way, but if they ever feel like it's a lost cause they'll just pull the Widevine lever.
- dTal 9y ago
  In theory, no not really. Everything required for your browser to obtain the stream must be provided in the webpage, so it's always possible for something else to pretend to be a browser and get ahold of the stream. In practice, they could combat the practicality by various methods, like embedding the ads in the video stream, throttling the stream to hurt the use-case of downloading vs watching (they already do this I think).
- blockoperation 9y ago
  It would be easy for them to fingerprint it and block it at a server level, given that it uses some hardcoded headers (which are probably sent in the wrong order versus the browser it's spoofing), doesn't fetch any of the images/stylesheets/etc on the page, and probably fetches scripts/manifests/etc in a predictable order that differs from YouTube's own scripts. Maybe they already do this (fingerprinting and logging, that is), but I haven't heard of anyone being banned for it, so it's probably not something to worry about. It would also be easy for them to just break the extraction code. The old code used to break every time the signature function changed, and while the current code solves that problem, there are still so many things that they could do to break it, and yet they don't (the current code has only broken a few times that I can think of, and I don't think any of those were intentional on Google's part). Technically it could violate these parts of the ToS, but they're all grey areas: - access through anything than the site or 'approved' clients (but youtube-dl does use the site, so it could just be classed as another user agent) - running automated services against them (running youtube-dl manually is probably fine, even for whole playlists or channels, but running a 'youtube-dl as a service' site like the one in this case is almost certainly not) - downloading videos (but youtube-dl can also be used for streaming, despite the name) I'm guessing that Google simply doesn't give a shit, as long as you're not using using it abusively (e.g. offering it as a service or using it to do mass-scraping).
Aissen 9y ago
This is not necessarily permanent. Does youtube-dl work with your Netflix subscription ? This shows the direction we're heading. PS: I know DRMs are broken eventually. But Netflix seems to be holding its ground for now, and seems to be attacked only through HDCP (the weakest link for now since you can find capture HDMI cards with stolen keys) for the highest-quality content.
crippleware 9y ago
With EME implemented across all major browsers and Google having made it impossible to disable Widevine in Chrome, I'd be amazed if that doesn't change in less than a year.
m-p-3 9y ago
And it works with so much more than YouTube. One of my favorite tools for media acquisition.

So glad I exist in a timeline where youtube-dl exists. Having culture locked up in these corporate sites that can wipe it out or remove it in less than a second makes me so anxious.

api 9y ago

I am a confirmed packrat. When I hear a really interesting podcast, obscure piece of music, etc. I mirror it to local storage... especially if it's political. Storage is cheap. Doesn't even have to be 'verboten' political. It would be easy for an AI bot trained to curtail e.g. 'hate speech' to delete loads of things that are just unorthodox.

Dirlewanger 9y ago

Agreed, but it also makes me a little sad that we could potentially only have some shitty compressed transcode of a transcode copy of some rare album, since YouTube is plagued with those kinds of uploads. Really makes the loss of what.cd hurt even more.

isatty 9y ago
red isn't doing too badly though
- ams6110 9y ago
  I'm a YouTube Red subscriber, and I also use youtube-dl to allow for offline listening on various devices. Since I'm paying Youtube, and not redistributing my downloads, I don't feel this is any different from DVR-ing a TV show or the old (and legal) practice of copying a vinyl LP to cassette tape so I could listen to it in my car.
  - baldfat 9y ago
    Except the content makers aren't getting revenue since YouTube Red pays for engagement. I LOVE YouTube Red I only use youtube-dl for special use cases.
  - gue5t 9y ago
    It seems likely that the gp is talking about what.cd successor redacted.ch, not the "YouTube Red" paid streaming service.
- lfam 9y ago
  Red is great, but if we want to preserve our musical heritage, we can't need to hide as we do it. I think that secret libraries are not as resilient as public archives.
intoverflow2 9y ago
Only enthusiasts can truly be trusted with archiving culture. Compare what.cd to iTunes. Compare Karagarga to Netflix, people actually go out of their way to create the best version of a movie, maybe the western release has incorrect hardcoded subs and awful picture quality. Someone goes out of their way to find the best version of it which may be something obscure like a Japanese laserdisc, rip it, fix subtitles and audio etc and release it for no monetary gain just the pure love of the media. People actually being paid to release this stuff care far less about it than enthusiasts.

hanklazard 9y ago

Agree completely. I am late to the party as I only heard about this tool a few days ago (from HN, of course), but I'm already archiving some of my favorite obscure videos. Props to the developers.

shortoncash 9y ago

I feel like things will change in the next few years. One of these decentralized video or storage services might take off and change things forever. Perhaps it's wishful thinking, but ever since blockchain technology got rapid adoption (at least among technical people), I can't help but think we're a few user interface innovations on top of decentralized technology away from really seeing a completely different kind of internet.

unlmtd1 9y ago
It's already there. IPFS. Just need a way to index it. There's Ethereum. Tezos could potentially do it cheaper.
- TeMPOraL 9y ago
  And a way to ensure you don't get sued out of your livelihood for the copyrighted file IPFS happened to cache on your machine.
azeirah 9y ago
I wonder if you could take a middle step, and host a community-run server for youtube-dl itself; run YouTube-mp3 on an anonymous and decentralized server. Perhaps Ethereum is the right technology for this. Would something like that be possible with Ethereum or otherwise blockchains in general?
- pavel_lishin 9y ago
  I don't understand blockchain technology in depth, but my understanding is that writing to the blockchain is usually not particularly cheap or fast, especially when writing large amounts of data. A blockchain that fills up with terabytes of pirated movies or mp3s would become difficult to operate, wouldn't it?
  - saalweachter 9y ago
    You'd be using some sort of filecoin-like storage system, where you pay people for distributed storage. If I've been following the whole Ethereum thing correctly, as an outsider who kind of looks down on it, this is right up it's alley. You write some sort of smart contract which allows for the uploading and downloading of music blobs, charges for the service, and if insufficient coins are garnered for storage deletes the least used files. I don't think you need a server or any such thing, and the only way to force a take down would be to take down the underlying services.

ocdtrekkie 9y ago

Out of curiosity: Is youtube-dl something that can be prevented from working? Does a download from youtube-dl look different to a Google server than a legitimate view? I guess it's kinda surprising to me that it still works after all this time, that there isn't some sort of authentication involved to prevent it.

Mindwipe 9y ago
YouTube has broken youtube-dl multiple times (a lot). It's a cat and mouse situation. Ultimately YouTube could start properly DRM-ing all their streams. If you watch a film you've bought from Google Play via YouTube it is protected by Widevine DRM, and youtube-dl fails. It seems like YouTube feel it's cheaper at the moment to try and frustrate downloading this way, but if they ever feel like it's a lost cause they'll just pull the Widevine lever.
dTal 9y ago
In theory, no not really. Everything required for your browser to obtain the stream must be provided in the webpage, so it's always possible for something else to pretend to be a browser and get ahold of the stream. In practice, they could combat the practicality by various methods, like embedding the ads in the video stream, throttling the stream to hurt the use-case of downloading vs watching (they already do this I think).
blockoperation 9y ago
It would be easy for them to fingerprint it and block it at a server level, given that it uses some hardcoded headers (which are probably sent in the wrong order versus the browser it's spoofing), doesn't fetch any of the images/stylesheets/etc on the page, and probably fetches scripts/manifests/etc in a predictable order that differs from YouTube's own scripts. Maybe they already do this (fingerprinting and logging, that is), but I haven't heard of anyone being banned for it, so it's probably not something to worry about. It would also be easy for them to just break the extraction code. The old code used to break every time the signature function changed, and while the current code solves that problem, there are still so many things that they could do to break it, and yet they don't (the current code has only broken a few times that I can think of, and I don't think any of those were intentional on Google's part). Technically it could violate these parts of the ToS, but they're all grey areas: - access through anything than the site or 'approved' clients (but youtube-dl does use the site, so it could just be classed as another user agent) - running automated services against them (running youtube-dl manually is probably fine, even for whole playlists or channels, but running a 'youtube-dl as a service' site like the one in this case is almost certainly not) - downloading videos (but youtube-dl can also be used for streaming, despite the name) I'm guessing that Google simply doesn't give a shit, as long as you're not using using it abusively (e.g. offering it as a service or using it to do mass-scraping).

Aissen 9y ago

This is not necessarily permanent. Does youtube-dl work with your Netflix subscription ? This shows the direction we're heading. PS: I know DRMs are broken eventually. But Netflix seems to be holding its ground for now, and seems to be attacked only through HDCP (the weakest link for now since you can find capture HDMI cards with stolen keys) for the highest-quality content.

crippleware 9y ago

With EME implemented across all major browsers and Google having made it impossible to disable Widevine in Chrome, I'd be amazed if that doesn't change in less than a year.

m-p-3 9y ago

And it works with so much more than YouTube. One of my favorite tools for media acquisition.