4 ms·

You don't need to scrap the page, YouTube provides the RSS stream for each channel: `https://www.youtube.com/feeds/videos.xml?channel_id=[ID]` https://www.youtu

• latexr 9y ago
  > You don't need to scrap the page They do if they are scraping playlists. Youtube assumes that when creators make a playlist, they’ll add videos to the top. As such, the RSS for a playlist[1] always returns just the top 15 results. But a lot of creators — typically the ones that take greater care into organising their playlists, such as Crash Course[2] — add videos to the bottom of their playlists. Those RSS feeds will be stuck at the same 15 items forever, and your only option is to scrape the page. [1]: https://www.youtube.com/feeds/videos.xml?playlist_id={playlist_id} https://www.youtube.com/feeds/videos.xml?playlist_id={playli... [2]: https://www.youtube.com/user/crashcourse https://www.youtube.com/user/crashcourse
  • zaarn 9y ago
    You can use youtube-dl's archiving feature for that. I use it regularly on my fairly large music playlist and it goes through all 20 pages just fine despite adding stuff on the bottom occasionally (I sorted it by creation date)