One of the websites where one can find this annnoying "infinite scroll" is YouTube channels. I wrote a quick and dirty script to address this annoyance. It ca

One of the websites where one can find this annnoying "infinite scroll" is YouTube channels. I wrote a quick and dirty script to address this annoyance. It can be used to output a table of all the video urls and video titles for any YouTube channel. "yy032" and "yy025" are some utilities I wrote to decode html and transform urls to HTTP for HTTP/1.1 pipelining, respectively.1 Instead of using yy025 and openssl, one could alternatively make a separate TCP connection for each HTTP request, e.g., using something like curl. Personally, I prefer not to make lots of connections when retrieving mutiple pages from the same domain. Here is a hypothetical example of how to use the script, "1.sh", to make a table of all the video urls and video titles in a channel. echo https://www.youtube.com/user/example/videos|sh 1.sh|yy025|openssl s_client -connect www.youtube.com:443 > 2.html sh 1.sh urls < 2.html > example.1 sh 1.sh titles < 2.html > example.2 rm 2.html paste -d '\t' example.1 example.2 # 1.sh case $1 in "") exec 2>/dev/null export Connection=close yy025|tcs www.youtube.com |sed 's/%25/%/g'|yy032 > 1.html while true;do x=$(sed 's/%25/%/g;s/\\//g' 1.html|yy032|grep -o "[^\"]*browse_ajax[^\"\\]*"|sed 's/u0026amp;/\&/g;s/&direct_render=1//;s,^,https://www.youtube.com,'); echo > 1.html; test ${#x} -gt 100||break echo "$x" echo "$x"|yy025|openssl s_client -connect www.youtube.com:443 -ign_eof > 1.html done; rm 1.html; ;;-h|-?|-help|--help) echo usage: echo https://www.youtube.com/user/example/videos \|$0 ;echo usage: $0 "[1|2]" \< 2.htm ;;1) sed 's/\\//g;s/u0026amp;//g;s/u0026quot;//g;s/u0026#39;//g'|grep -o "ltr\" title=\"[^\"]*"|sed 's/ltr..title=.//' ;;2) sed 's/\\//g;s/u0026amp;//g;s/u0026quot;//g'|grep -o "[^\"]*watch?v=[^\"]*"|sed 's,^,https://www.youtube.com,'|uniq esac 1 https://news.ycombinator.com/item?id=17689165 https://news.ycombinator.com/item?id=17689165 https://news.ycombinator.com/item?id=17689152 https://news.ycombinator.com/item?id=17689152