4 ms·

With video and other rich media ? You will need hundreds of thousands or even millions of these disks. YT alone adds 500hrs of video per min . That is at minim

With video and other rich media ? You will need hundreds of thousands or even millions of these disks. YT alone adds 500hrs of video per min . That is at minimum 9000 TB / day (500 18TB disks/day - 180,000 disks a year) [1] If YT is no more than say 1% of the public internet (Facebook is probably storing lot more , most of the data is not public though, Instagram is ) Then you are looking probably at 18 million disks a year without any backups and copies considered . [1] it is assuming SD quality 220 mb per hour single format and one copy.

exikyut 6y ago
> YT alone adds 500hrs of video per min . That is at minimum 9000 TB / day (500 18TB disks/day ...) ... assuming SD quality 220 mb per hour single format and one copy. Nitpick: Each bigger resolution stacks on top of the previous one, so 8K60fps means storing 8K60 (webm+av1), alongside 4K60 HDR (webm), 4K60 (webm+av1), 4K (webm), 1440p60 HDR (webm), 1440p60 (webm+av1), 1440p (webm), 1080p60 HDR (webm), 1080p60 (webm+h264+av1), 1080p (webm+h264), 720p60 HDR (webm), 720p60 (webm+h264+av1), 720p (webm+h264), 480p60 HDR (webm), 480p (webm+h264+av1), 360p60 HDR (webm), 360p (webm+h264+av1), 240p60 HDR (webm), 240p (webm+h264+av1), 144p60 HDR (webm), and 144p (webm+h264+av1). And as noted, a given resolution is often stored in multiple video formats, so there's that too. The data needed to store a single video thus adds up quickly: $ youtube-dl -F 1La4QzGeaaQ | grep -o '[0-9\.]\+.iB' | sed 's/MiB/*(1024^2)/;s/GiB/*(1024^3)/;s/.*/(&)/' \ | tr '\n' + | sed 's/$.*$+$/scale=3;print(\1)\/(1024^3),"G\n"\n/' | bc -ql 8.423G So for https://youtu.be/1La4QzGeaaQ https://youtu.be/1La4QzGeaaQ (a random 8K video I found), storing 5 minutes 37 seconds requires about 8.5GB due to all the transcoding. It's quite possible YouTube also keeps the original uploads around for a bit as well. --- It incidentally proved so complicated to correctly type out the format list above manually it was actually easier to figure out the following regex: $ youtube-dl -F 1La4QzGeaaQ | sed -n '/video/{s/^.\{35\}//;s/ \+[0-9]\+k \+, \+/ (/;s/webm.*\|vp9.*/webm/;s/avc.*/h264/;s/av01.*/av1/;s/4320p/8K/;s/2160p/4K/;p}' \ | sort -hr | sed ':1;H;N;${:2;s/$[^\(]\+$($[^\n]\+$\n$\1$(/\1(\2+/;t2;s/$\n\|$$/), /g;s/, $//};b1' (The above took probably twice as long to type out as the list it produced, but it was a lot more fun, and importantly verified its own correctness, which I was having a hard time doing manually. Plus, it's cool to make sed do new things - like token deduplication and newline coalescence in this case (remove the 2nd sed command to see what I mean).)

exikyut 6y ago

> YT alone adds 500hrs of video per min . That is at minimum 9000 TB / day (500 18TB disks/day ...) ... assuming SD quality 220 mb per hour single format and one copy. Nitpick: Each bigger resolution stacks on top of the previous one, so 8K60fps means storing 8K60 (webm+av1), alongside 4K60 HDR (webm), 4K60 (webm+av1), 4K (webm), 1440p60 HDR (webm), 1440p60 (webm+av1), 1440p (webm), 1080p60 HDR (webm), 1080p60 (webm+h264+av1), 1080p (webm+h264), 720p60 HDR (webm), 720p60 (webm+h264+av1), 720p (webm+h264), 480p60 HDR (webm), 480p (webm+h264+av1), 360p60 HDR (webm), 360p (webm+h264+av1), 240p60 HDR (webm), 240p (webm+h264+av1), 144p60 HDR (webm), and 144p (webm+h264+av1). And as noted, a given resolution is often stored in multiple video formats, so there's that too. The data needed to store a single video thus adds up quickly: $ youtube-dl -F 1La4QzGeaaQ | grep -o '[0-9\.]\+.iB' | sed 's/MiB/*(1024^2)/;s/GiB/*(1024^3)/;s/.*/(&)/' \ | tr '\n' + | sed 's/$.*$+$/scale=3;print(\1)\/(1024^3),"G\n"\n/' | bc -ql 8.423G So for https://youtu.be/1La4QzGeaaQ https://youtu.be/1La4QzGeaaQ (a random 8K video I found), storing 5 minutes 37 seconds requires about 8.5GB due to all the transcoding. It's quite possible YouTube also keeps the original uploads around for a bit as well. --- It incidentally proved so complicated to correctly type out the format list above manually it was actually easier to figure out the following regex: $ youtube-dl -F 1La4QzGeaaQ | sed -n '/video/{s/^.\{35\}//;s/ \+[0-9]\+k \+, \+/ (/;s/webm.*\|vp9.*/webm/;s/avc.*/h264/;s/av01.*/av1/;s/4320p/8K/;s/2160p/4K/;p}' \ | sort -hr | sed ':1;H;N;${:2;s/$[^\(]\+$($[^\n]\+$\n$\1$(/\1(\2+/;t2;s/$\n\|$$/), /g;s/, $//};b1' (The above took probably twice as long to type out as the list it produced, but it was a lot more fun, and importantly verified its own correctness, which I was having a hard time doing manually. Plus, it's cool to make sed do new things - like token deduplication and newline coalescence in this case (remove the 2nd sed command to see what I mean).)