4 ms·

And yt-dlp uses an 'Android API' to stop throttling

by vmoore 4y ago

heavyset_go 4y ago
I expect the ability to use APIs like this will be hindered once remote attestation becomes the norm.
- cute_boi 4y ago
  This is reason why we must diversify these techs. If same company builds everything like mobile, os, search engine, Youtube, it would have debacle in future. If there is diversification, it simply takes toil and sweat to convince everyone.
1vuio0pswjnm7 4y ago
One has to execute functions from base.js^1 to modify the "n" URL parameter and the "sig" parameter to get the fastest download speeds. (One can still download videos with the original n parameter, or without the n parameter, but download speeds will be slower.) By itself, using the ANDROID API instead of the WEB API^2 does nothing to affect download speeds. I can block yt-dlp's POST request indicating the client and API name and this has no effect on download speed. A website that forces users to run Javascript in order to get faster download speeds. This is not a new idea. 1. https://www.youtube.com/s/player/{player_version}/player_ias.vflset/en_US/base.js https://www.youtube.com/s/player/{player_version}/player_ias... 2. From the YouTube video page JSON: ANDROID API KEY AIzaSyA8eiZmM1FaDVjRy-df2KTyQ_vz_yYM39w WEB API KEY AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8
- 1vuio0pswjnm7 4y ago
  yt-dl and yt-dlp use Python to interpet the Javascript functions that modify "n" and "sig", but a faster language could be used. For example, V8 is written in C++. One of the reasons yt-dl and yt-dlp are so slow is because they do (too) many do other things besides running the Javascript, modifying n and sig, and spitting out a fast download URL. Before they can run the JS, the video page needs to be retrieved, but waiting for Python to start up and do this is slow. A YouTube video page can be retrieved much faster using netcat, outside of Python. YouTube video can be downloaded much faster using an HTTP client like tnftp, directly, outside of Python. YouTube video can be converted much faster using ffmpeg, directly, outside of Python. And so on. These programs start instantaniously when compared to the slow start up time of Python. The Python startup latency is unbearable. At the very least yt-dl and yt-dlp should accept an already downloaded YouTube video page as input instead of forcing the user to use Python (or Python calling another program) to download the page. The recalculation of "n" and "sig" do not have to occur within seconds of retrieving the video page. There is no need to require the user to use Python for downloading webpages. The values in the YouTube video page are good for a substantial amount of time. Both scripts have an option to output the new n and sig values in a download URL or JSON containing the download URLs, so the user can use an HTTP client directly, outside of Python. But the user still has to use Python to download each video page. Using an HTTP client directly would be faster. Using yt-dl and yt-dlp just to output optimal download URLs feels like overkill.
  - Scaevolus 4y ago
    If you find yt-dlp particularly slow, you can use aria2c to perform the download itself. It doesn't solve any startup latency problems, but it generally can do the HTTP requests faster.
  - marcan_42 4y ago
    This is complete nonsense. The overwhelming majority of video download time is spent transferring the actual video data, limited by throttling or network speed. The time it takes for these tools to start up and deal with the API/JS stuff is inconsequential for any video longer than a minute or so. And even then most of the time is network latency for the API stuff, not local processing. Netcat isn't going to go any faster. Absolutely nobody thinks optimizing the meta/API processing of yt-dlp & co is worth it. This is exactly why we have high level programming languages that make all of this much easier, instead of trying to write HTML and JS parsing in plain C. Keep in mind these tools support dozens or hundreds or websites, not just YouTube. If you think rewriting yt-dlp in C is worth it, go right ahead, but you're not going to make it significantly faster; you're just going to make maintenance a much bigger pain for yourself. Pick the right tool for the job. Python is absolutely (one of) the right tools for this job. (For the record: I use C and Python on a daily basis, and will use whatever language is appropriate for each situation.)
    - 1vuio0pswjnm7 4y ago
      "Python is absolutely (one of) the right tools for this job." This opinion assumes it is a single job. I see multiple jobs. The number "options" provided by yt-dl(p) gives us a clue. There is nothing wrong with preferring to use larger, more complicated, "multi-purpose" utilities. There will always be plenty to choose from. However the idea of using smaller, less complicated, single purpose utilities is not "nonsense". It makes sense in many cases and some users may prefer it. The statements I make about speed are from day-to-day experience not conjecture.
  - Retr0id 4y ago
    I look forward to seeing your implementation of this solution.
deleted 4y ago
[deleted]

And yt-dlp uses an 'Android API' to stop throttling

by vmoore 4y ago

And yt-dlp uses an 'Android API' to stop throttling

heavyset_go 4y ago

I expect the ability to use APIs like this will be hindered once remote attestation becomes the norm.

cute_boi 4y ago
This is reason why we must diversify these techs. If same company builds everything like mobile, os, search engine, Youtube, it would have debacle in future. If there is diversification, it simply takes toil and sweat to convince everyone.

1vuio0pswjnm7 4y ago

One has to execute functions from base.js^1 to modify the "n" URL parameter and the "sig" parameter to get the fastest download speeds. (One can still download videos with the original n parameter, or without the n parameter, but download speeds will be slower.) By itself, using the ANDROID API instead of the WEB API^2 does nothing to affect download speeds. I can block yt-dlp's POST request indicating the client and API name and this has no effect on download speed. A website that forces users to run Javascript in order to get faster download speeds. This is not a new idea. 1. https://www.youtube.com/s/player/{player_version}/player_ias.vflset/en_US/base.js https://www.youtube.com/s/player/{player_version}/player_ias... 2. From the YouTube video page JSON: ANDROID API KEY AIzaSyA8eiZmM1FaDVjRy-df2KTyQ_vz_yYM39w WEB API KEY AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8

1vuio0pswjnm7 4y ago
yt-dl and yt-dlp use Python to interpet the Javascript functions that modify "n" and "sig", but a faster language could be used. For example, V8 is written in C++. One of the reasons yt-dl and yt-dlp are so slow is because they do (too) many do other things besides running the Javascript, modifying n and sig, and spitting out a fast download URL. Before they can run the JS, the video page needs to be retrieved, but waiting for Python to start up and do this is slow. A YouTube video page can be retrieved much faster using netcat, outside of Python. YouTube video can be downloaded much faster using an HTTP client like tnftp, directly, outside of Python. YouTube video can be converted much faster using ffmpeg, directly, outside of Python. And so on. These programs start instantaniously when compared to the slow start up time of Python. The Python startup latency is unbearable. At the very least yt-dl and yt-dlp should accept an already downloaded YouTube video page as input instead of forcing the user to use Python (or Python calling another program) to download the page. The recalculation of "n" and "sig" do not have to occur within seconds of retrieving the video page. There is no need to require the user to use Python for downloading webpages. The values in the YouTube video page are good for a substantial amount of time. Both scripts have an option to output the new n and sig values in a download URL or JSON containing the download URLs, so the user can use an HTTP client directly, outside of Python. But the user still has to use Python to download each video page. Using an HTTP client directly would be faster. Using yt-dl and yt-dlp just to output optimal download URLs feels like overkill.
- Scaevolus 4y ago
  If you find yt-dlp particularly slow, you can use aria2c to perform the download itself. It doesn't solve any startup latency problems, but it generally can do the HTTP requests faster.
- marcan_42 4y ago
  This is complete nonsense. The overwhelming majority of video download time is spent transferring the actual video data, limited by throttling or network speed. The time it takes for these tools to start up and deal with the API/JS stuff is inconsequential for any video longer than a minute or so. And even then most of the time is network latency for the API stuff, not local processing. Netcat isn't going to go any faster. Absolutely nobody thinks optimizing the meta/API processing of yt-dlp & co is worth it. This is exactly why we have high level programming languages that make all of this much easier, instead of trying to write HTML and JS parsing in plain C. Keep in mind these tools support dozens or hundreds or websites, not just YouTube. If you think rewriting yt-dlp in C is worth it, go right ahead, but you're not going to make it significantly faster; you're just going to make maintenance a much bigger pain for yourself. Pick the right tool for the job. Python is absolutely (one of) the right tools for this job. (For the record: I use C and Python on a daily basis, and will use whatever language is appropriate for each situation.)
  - 1vuio0pswjnm7 4y ago
    "Python is absolutely (one of) the right tools for this job." This opinion assumes it is a single job. I see multiple jobs. The number "options" provided by yt-dl(p) gives us a clue. There is nothing wrong with preferring to use larger, more complicated, "multi-purpose" utilities. There will always be plenty to choose from. However the idea of using smaller, less complicated, single purpose utilities is not "nonsense". It makes sense in many cases and some users may prefer it. The statements I make about speed are from day-to-day experience not conjecture.
- Retr0id 4y ago
  I look forward to seeing your implementation of this solution.

deleted 4y ago

[deleted]