7 ms·

Beginner dev looks at how YouTube sends videos

150 points by vsgherzi 3y ago

swyx 3y ago
i picked a random video and noticed that the audio (mp3) download was the same filesize as the video (mp4) download. this seems wrong. some thing to do in the encoding?
- justinclift 3y ago
  Maybe it's sending the same bitstream for both?
  - c1ncy28 3y ago
    He's just downloading both the aac audio and h264 video stream, and then renaming the resulting mp4 file to .mp3
vsgherzi 3y ago
Super early on in my carrier. Just wanted to download an mp3 from my favorite artist without weird ads or tracking
- geraldhh 3y ago
  sure thing, who doesn't
- SushiHippie 3y ago
  I'd recommend you to have a look at yt-dlp [0] and NewPipe extractor [1] [0] https://github.com/yt-dlp/yt-dlp https://github.com/yt-dlp/yt-dlp [1] https://teamnewpipe.github.io/documentation/ https://teamnewpipe.github.io/documentation/
- ramesh31 3y ago
  You got the knack, bud, and that's all it takes. Plenty of people have finished CS degrees without acquiring the knack. But that's really all it takes. If you've got it, it's just a matter of time before you're knocking people's socks off.
  - swyx 3y ago
    i find it interesting how some HN threads are super supportive like this one and others tear people apart for overstating their expertise, getting overleveled job titles as a "senior" developer while missing some esoteric/deep knowledge. we're all on our meandering paths somehow.
    - ramesh31 3y ago
      After spending enough years mentoring juniors, you pick up on it pretty fast whether someone is worth your time. Ask for a specific solution, and you'll get it from me once. Ask for understanding the generalized case, and I'll happily sit around explaining things to you all day long. Even asking for a specific solution is fine if it's followed up by thoughtful probing questions on the general patterns. But there's just no room for hand holding in this industry.
    - 100721 3y ago
      It's useful to remember that this is a community of individuals, each with their own opinions and experiences.
    - sevagh 3y ago
      There's a line of "real curiosity about interesting aspects of computer software" and no-substance snake oil. So, in one case, it's "low substance but good effort", in the other it's "low substance and low effort"
- np1810 3y ago
  Well written article... nitpick - your article date [1] as of posting this comment is DEC 31 2024... > fullstack dev versus youtube - DEC 31 2024 [1] https://vincentsg.dev/blog/ https://vincentsg.dev/blog/
  - gverrilla 3y ago
    Also the coffee in the image is yellow, which is really weird.
NiekvdMaas 3y ago
Nice write-up! For a better maintainable version, I would recommend `yt-dlp --dump-json` instead. This will give you hundreds of other supported sites for free.
thecleaner 3y ago
Now that is how you spend new year. Also beginner dev ? Dont be so modest mate.
ramesh31 3y ago
This rules. Pull back the curtain and vanquish the black magic. We should all know how this stuff works, if only to protect ourselves.
sam1r 3y ago
>>> I quickly threw together an ugly frontend and posted it to digital ocean under an express server and prepped another project to repair my broken discord bot. That is quite some versatile lingo for a beginner dev.
- coolandsmartrr 3y ago
  The original post doesn't mention "beginner", so I'm not sure why OP (link poster) decided to editorialize it as such.
  - mikepurvis 3y ago
    Pretty sure this is a self-submission, based on the user's comments elsewhere in the thread.
    - deleted 3y ago
      [deleted]
    - codetrotter 3y ago
      Also their username and the email in their profile is similar to the domain name. Yes, I think the submitter is the author.
  - akoboldfrying 3y ago
    OP describes themselves as a "new developer": >Big shoutout to the invidious project and tyrrrz's blog they did basically all of the heavy lifting for this new developer!
  - marginalia_nu 3y ago
    To be fair, we could all probably benefit from regarding ourselves as beginners [1] [1] https://en.wikipedia.org/wiki/Shoshin https://en.wikipedia.org/wiki/Shoshin
- throwaway19dmfa 3y ago
  To be fair there are a lot of people who are familiar with tech stuff and have just never actually learnt any sort of "dev" properly before who I would consider as beginner. These people also tend up pick things up very quickly at the beginning and are able to use the right lingo after a few months of learning -- anyone who has done a few months of a bootcamp can talk/write like that even if they don't have a deep understanding of what they are talking about.
katella 3y ago
Is there a way to do this clientside?
- Daneel_ 3y ago
  yt-dlp is the project you want
tintedfireglass 3y ago
uhh the blog is wrongly dated 31st Dec 2024 instead of 2023.
account-5 3y ago
This is cool. I wish I could do this sort of thing, however when I come up again something like this I assume it's already been done and go look for the solution. In this case I'd likely have just used yt-dlp. However, had that project been dmca'd, by disreputable organisatios I'd likely have remained stuck, so kudos.
pocketarc 3y ago
The end of the post talks about hiring him, and honestly, if anyone’s hiring, here’s one heck of a bargain: Someone who’s just starting out but who’s already shown he’s perfectly capable of figuring stuff out by himself.
- darylteo 3y ago
  Definitely shows good intuition. Even if not doing the actual figuring out, still finds a possible solution and reverse engineers it. Forget 10x developer, this is the 10% of developers. Open to any ideas how this level of intuition can be imparted at scale...
  - rpmisms 3y ago
    It's a style of thinking. It can be taught, but mostly by example.
- otteromkram 3y ago
  Looks like they graduated college. >Oh... that's ALOT of parameters If they would just learn that "a lot" is two words. Given that their resume says Coding Ninjas tutor and Coding Ninjas is one of the worst websites out there for coding challenges and tutorials, while using incorrect syntax for Python and C++ (probably to lazily port problems from one language to another), this makes sense. I'll probably get downvoted for my comment because of all the "touchy feelys" around here, but a lot of people can do what OP did sans college. What makes them extra hireable? Because they had a hacker news story catch some action on a holiday weekend?
- pw6hv 3y ago
  If I was a hiring manager, I would certainly appreciate this attitude. However, I would value as well a person who would use an already made tool for doing the job (such as yt-dlp). Only problem is that the second type wouldn't write a blog post about it and get traction on HN :D
  - petepete 3y ago
arcza 3y ago
Thanks for posting and well done. Small suggestion for your writing style: use less ALL CAPS / BOOM! / WOW! styles and embrace less dramatic wording: it would make your site read more professionally and be easier to digest. Good luck.
- 243423443 3y ago
  I enjoyed the writing style.
  - ResNet 3y ago
    Same here, it felt authentic -- a really fun read.
- sevagh 3y ago
  Their tone reflects their excitement, it's authentic and great.
  - rad_gruchalski 3y ago
    No. What you get is this kind of communication in work email and all over slack. Boom, wow, a dozen random emojis and even the author has no clue what these emojis suppose to mean.
- withinboredom 3y ago
  It’s a blog, not a research paper.
  - deleted 3y ago
    [deleted]
mcny 3y ago
I tried a YouTube shorts video I saw yesterday that for some reason failed to download with yt dlp (also tried with newpipe) to see if this would behave differently. https://www.youtube.com/watch?v=JQgKhZZyBYg https://www.youtube.com/watch?v=JQgKhZZyBYg Something is squirly with this YouTube video. Can someone else give it a try?
- phantomathkg 3y ago
  try remove the url and just `yt-dlp JQgKhZZyBYg`
  - mcny 3y ago
    that worked. strangely enough. ---snip--- yt-dlp.exe --restrict-filenames --trim-filenames 30 --format "bestvideoext=mp4+bestaudioext=m4a/mp4" -i "JQgKhZZyBYg" Monday, January 1, 2024 10:46:45 AM [youtube] Extracting URL: JQgKhZZyBYg [youtube] JQgKhZZyBYg: Downloading webpage [youtube] JQgKhZZyBYg: Downloading ios player API JSON [youtube] JQgKhZZyBYg: Downloading android player API JSON [youtube] JQgKhZZyBYg: Downloading m3u8 information [info] JQgKhZZyBYg: Downloading 1 format(s): 22 [download] Resuming download at byte 130048 [download] Got error: 0 bytes read, 10462009 more expected. Retrying (1/10)... [download] Got error: 0 bytes read, 10052978 more expected. Retrying (2/10)... [download] Got error: 0 bytes read, 10373389 more expected. Retrying (3/10)... [download] Got error: 0 bytes read, 10107360 more expected. Retrying (4/10)... [download] Got error: 0 bytes read, 10307771 more expected. Retrying (5/10)... [download] Got error: 0 bytes read, 10202520 more expected. Retrying (6/10)... [download] Got error: 0 bytes read, 10455655 more expected. Retrying (7/10)... [download] Got error: 0 bytes read, 9968301 more expected. Retrying (8/10)... [download] Got error: 0 bytes read, 10027086 more expected. Retrying (9/10)... [download] Got error: 0 bytes read, 10392472 more expected. Retrying (10/10)... [download] Got error: 0 bytes read, 10022086 more expected. Giving up after 10 retries Monday, January 1, 2024 10:46:51 AM --- snip --- yt-dlp.exe "JQgKhZZyBYg" Monday, January 1, 2024 10:48:03 AM [youtube] Extracting URL: JQgKhZZyBYg [youtube] JQgKhZZyBYg: Downloading webpage [youtube] JQgKhZZyBYg: Downloading ios player API JSON [youtube] JQgKhZZyBYg: Downloading android player API JSON [youtube] JQgKhZZyBYg: Downloading m3u8 information [info] JQgKhZZyBYg: Downloading 1 format(s): 247+251 [download] Destination: Jet crashes at Thunder Over Michigan air show at Willow Run Airport [JQgKhZZyBYg].f247.webm [download] 100% of 8.37MiB in 00:00:00 at 17.44MiB/s [download] Destination: Jet crashes at Thunder Over Michigan air show at Willow Run Airport [JQgKhZZyBYg].f251.webm [download] 100% of 599.52KiB in 00:00:00 at 3.53MiB/s [Merger] Merging formats into "Jet crashes at Thunder Over Michigan air show at Willow Run Airport [JQgKhZZyBYg].webm" Deleting original file Jet crashes at Thunder Over Michigan air show at Willow Run Airport [JQgKhZZyBYg].f247.webm (pass -k to keep) Deleting original file Jet crashes at Thunder Over Michigan air show at Willow Run Airport [JQgKhZZyBYg].f251.webm (pass -k to keep) Monday, January 1, 2024 10:48:07 AM
xwowsersx 3y ago
What does the author mean that he "created a fetch request in node using the android example"? He means he made the request using Node and impersonated an Android client?
- chupapimunyenyo 3y ago
  Yes
orenlindsey 3y ago
Here's my favorite site for downloading YouTube videos. https://github.com/micahlt/vesper https://github.com/micahlt/vesper It's made by a friend of mine, I thought he had a writeup about it, but I can't find it.
arwineap 3y ago
There is basically nothing a beginner could learn with that is completely novel. We can probably assume this gentlemen already knew about yt-dl and it's derivatives; that wasn't the point of the exercise.
remram 3y ago
Is "fullstack developer" just "web developer"?

150 points by vsgherzi 3y ago

swyx 3y ago

i picked a random video and noticed that the audio (mp3) download was the same filesize as the video (mp4) download. this seems wrong. some thing to do in the encoding?

justinclift 3y ago
Maybe it's sending the same bitstream for both?
- c1ncy28 3y ago
  He's just downloading both the aac audio and h264 video stream, and then renaming the resulting mp4 file to .mp3

vsgherzi 3y ago

Super early on in my carrier. Just wanted to download an mp3 from my favorite artist without weird ads or tracking

geraldhh 3y ago
sure thing, who doesn't
SushiHippie 3y ago
I'd recommend you to have a look at yt-dlp [0] and NewPipe extractor [1] [0] https://github.com/yt-dlp/yt-dlp https://github.com/yt-dlp/yt-dlp [1] https://teamnewpipe.github.io/documentation/ https://teamnewpipe.github.io/documentation/
ramesh31 3y ago
You got the knack, bud, and that's all it takes. Plenty of people have finished CS degrees without acquiring the knack. But that's really all it takes. If you've got it, it's just a matter of time before you're knocking people's socks off.
- swyx 3y ago
  i find it interesting how some HN threads are super supportive like this one and others tear people apart for overstating their expertise, getting overleveled job titles as a "senior" developer while missing some esoteric/deep knowledge. we're all on our meandering paths somehow.
  - ramesh31 3y ago
    After spending enough years mentoring juniors, you pick up on it pretty fast whether someone is worth your time. Ask for a specific solution, and you'll get it from me once. Ask for understanding the generalized case, and I'll happily sit around explaining things to you all day long. Even asking for a specific solution is fine if it's followed up by thoughtful probing questions on the general patterns. But there's just no room for hand holding in this industry.
  - 100721 3y ago
    It's useful to remember that this is a community of individuals, each with their own opinions and experiences.
  - sevagh 3y ago
    There's a line of "real curiosity about interesting aspects of computer software" and no-substance snake oil. So, in one case, it's "low substance but good effort", in the other it's "low substance and low effort"
np1810 3y ago
Well written article... nitpick - your article date [1] as of posting this comment is DEC 31 2024... > fullstack dev versus youtube - DEC 31 2024 [1] https://vincentsg.dev/blog/ https://vincentsg.dev/blog/
- gverrilla 3y ago
  Also the coffee in the image is yellow, which is really weird.

NiekvdMaas 3y ago

Nice write-up! For a better maintainable version, I would recommend `yt-dlp --dump-json` instead. This will give you hundreds of other supported sites for free.

thecleaner 3y ago

Now that is how you spend new year. Also beginner dev ? Dont be so modest mate.

ramesh31 3y ago

This rules. Pull back the curtain and vanquish the black magic. We should all know how this stuff works, if only to protect ourselves.

sam1r 3y ago

>>> I quickly threw together an ugly frontend and posted it to digital ocean under an express server and prepped another project to repair my broken discord bot. That is quite some versatile lingo for a beginner dev.

coolandsmartrr 3y ago
The original post doesn't mention "beginner", so I'm not sure why OP (link poster) decided to editorialize it as such.
- mikepurvis 3y ago
  Pretty sure this is a self-submission, based on the user's comments elsewhere in the thread.
  - deleted 3y ago
    [deleted]
  - codetrotter 3y ago
    Also their username and the email in their profile is similar to the domain name. Yes, I think the submitter is the author.
- akoboldfrying 3y ago
  OP describes themselves as a "new developer": >Big shoutout to the invidious project and tyrrrz's blog they did basically all of the heavy lifting for this new developer!
- marginalia_nu 3y ago
  To be fair, we could all probably benefit from regarding ourselves as beginners [1] [1] https://en.wikipedia.org/wiki/Shoshin https://en.wikipedia.org/wiki/Shoshin
throwaway19dmfa 3y ago
To be fair there are a lot of people who are familiar with tech stuff and have just never actually learnt any sort of "dev" properly before who I would consider as beginner. These people also tend up pick things up very quickly at the beginning and are able to use the right lingo after a few months of learning -- anyone who has done a few months of a bootcamp can talk/write like that even if they don't have a deep understanding of what they are talking about.

katella 3y ago

Is there a way to do this clientside?

Daneel_ 3y ago
yt-dlp is the project you want

tintedfireglass 3y ago

uhh the blog is wrongly dated 31st Dec 2024 instead of 2023.

account-5 3y ago

This is cool. I wish I could do this sort of thing, however when I come up again something like this I assume it's already been done and go look for the solution. In this case I'd likely have just used yt-dlp. However, had that project been dmca'd, by disreputable organisatios I'd likely have remained stuck, so kudos.

pocketarc 3y ago

The end of the post talks about hiring him, and honestly, if anyone’s hiring, here’s one heck of a bargain: Someone who’s just starting out but who’s already shown he’s perfectly capable of figuring stuff out by himself.

darylteo 3y ago
Definitely shows good intuition. Even if not doing the actual figuring out, still finds a possible solution and reverse engineers it. Forget 10x developer, this is the 10% of developers. Open to any ideas how this level of intuition can be imparted at scale...
- rpmisms 3y ago
  It's a style of thinking. It can be taught, but mostly by example.
otteromkram 3y ago
Looks like they graduated college. >Oh... that's ALOT of parameters If they would just learn that "a lot" is two words. Given that their resume says Coding Ninjas tutor and Coding Ninjas is one of the worst websites out there for coding challenges and tutorials, while using incorrect syntax for Python and C++ (probably to lazily port problems from one language to another), this makes sense. I'll probably get downvoted for my comment because of all the "touchy feelys" around here, but a lot of people can do what OP did sans college. What makes them extra hireable? Because they had a hacker news story catch some action on a holiday weekend?
pw6hv 3y ago
If I was a hiring manager, I would certainly appreciate this attitude. However, I would value as well a person who would use an already made tool for doing the job (such as yt-dlp). Only problem is that the second type wouldn't write a blog post about it and get traction on HN :D
- petepete 3y ago

arcza 3y ago

Thanks for posting and well done. Small suggestion for your writing style: use less ALL CAPS / BOOM! / WOW! styles and embrace less dramatic wording: it would make your site read more professionally and be easier to digest. Good luck.

243423443 3y ago
I enjoyed the writing style.
- ResNet 3y ago
  Same here, it felt authentic -- a really fun read.
sevagh 3y ago
Their tone reflects their excitement, it's authentic and great.
- rad_gruchalski 3y ago
  No. What you get is this kind of communication in work email and all over slack. Boom, wow, a dozen random emojis and even the author has no clue what these emojis suppose to mean.
withinboredom 3y ago
It’s a blog, not a research paper.
- deleted 3y ago
  [deleted]

mcny 3y ago

I tried a YouTube shorts video I saw yesterday that for some reason failed to download with yt dlp (also tried with newpipe) to see if this would behave differently. https://www.youtube.com/watch?v=JQgKhZZyBYg https://www.youtube.com/watch?v=JQgKhZZyBYg Something is squirly with this YouTube video. Can someone else give it a try?

phantomathkg 3y ago
try remove the url and just `yt-dlp JQgKhZZyBYg`
- mcny 3y ago
  that worked. strangely enough. ---snip--- yt-dlp.exe --restrict-filenames --trim-filenames 30 --format "bestvideoext=mp4+bestaudioext=m4a/mp4" -i "JQgKhZZyBYg" Monday, January 1, 2024 10:46:45 AM [youtube] Extracting URL: JQgKhZZyBYg [youtube] JQgKhZZyBYg: Downloading webpage [youtube] JQgKhZZyBYg: Downloading ios player API JSON [youtube] JQgKhZZyBYg: Downloading android player API JSON [youtube] JQgKhZZyBYg: Downloading m3u8 information [info] JQgKhZZyBYg: Downloading 1 format(s): 22 [download] Resuming download at byte 130048 [download] Got error: 0 bytes read, 10462009 more expected. Retrying (1/10)... [download] Got error: 0 bytes read, 10052978 more expected. Retrying (2/10)... [download] Got error: 0 bytes read, 10373389 more expected. Retrying (3/10)... [download] Got error: 0 bytes read, 10107360 more expected. Retrying (4/10)... [download] Got error: 0 bytes read, 10307771 more expected. Retrying (5/10)... [download] Got error: 0 bytes read, 10202520 more expected. Retrying (6/10)... [download] Got error: 0 bytes read, 10455655 more expected. Retrying (7/10)... [download] Got error: 0 bytes read, 9968301 more expected. Retrying (8/10)... [download] Got error: 0 bytes read, 10027086 more expected. Retrying (9/10)... [download] Got error: 0 bytes read, 10392472 more expected. Retrying (10/10)... [download] Got error: 0 bytes read, 10022086 more expected. Giving up after 10 retries Monday, January 1, 2024 10:46:51 AM --- snip --- yt-dlp.exe "JQgKhZZyBYg" Monday, January 1, 2024 10:48:03 AM [youtube] Extracting URL: JQgKhZZyBYg [youtube] JQgKhZZyBYg: Downloading webpage [youtube] JQgKhZZyBYg: Downloading ios player API JSON [youtube] JQgKhZZyBYg: Downloading android player API JSON [youtube] JQgKhZZyBYg: Downloading m3u8 information [info] JQgKhZZyBYg: Downloading 1 format(s): 247+251 [download] Destination: Jet crashes at Thunder Over Michigan air show at Willow Run Airport [JQgKhZZyBYg].f247.webm [download] 100% of 8.37MiB in 00:00:00 at 17.44MiB/s [download] Destination: Jet crashes at Thunder Over Michigan air show at Willow Run Airport [JQgKhZZyBYg].f251.webm [download] 100% of 599.52KiB in 00:00:00 at 3.53MiB/s [Merger] Merging formats into "Jet crashes at Thunder Over Michigan air show at Willow Run Airport [JQgKhZZyBYg].webm" Deleting original file Jet crashes at Thunder Over Michigan air show at Willow Run Airport [JQgKhZZyBYg].f247.webm (pass -k to keep) Deleting original file Jet crashes at Thunder Over Michigan air show at Willow Run Airport [JQgKhZZyBYg].f251.webm (pass -k to keep) Monday, January 1, 2024 10:48:07 AM

xwowsersx 3y ago

What does the author mean that he "created a fetch request in node using the android example"? He means he made the request using Node and impersonated an Android client?

chupapimunyenyo 3y ago
Yes

orenlindsey 3y ago

Here's my favorite site for downloading YouTube videos. https://github.com/micahlt/vesper https://github.com/micahlt/vesper It's made by a friend of mine, I thought he had a writeup about it, but I can't find it.

arwineap 3y ago

There is basically nothing a beginner could learn with that is completely novel. We can probably assume this gentlemen already knew about yt-dl and it's derivatives; that wasn't the point of the exercise.

remram 3y ago

Is "fullstack developer" just "web developer"?