5 ms·
I once asked one of the original YouTube infra engineers “will you ever need to delete the long tail of videos no one watches” They said it didn’t matter, beca
by repeekad 8mo ago
I once asked one of the original YouTube infra engineers “will you ever need to delete the long tail of videos no one watches”
They said it didn’t matter, because the sheer volume of new data flowing in growing so fast made the old data just a drop in the bucket
- wasmainiac 8mo agoI wonder if that still holds true? The volume of videos increases exponentially especially with AI slop, I wonder if at some point they will have to limit the storage per user, with a paid model if you surpass that limit. Many people who upload many videos I guess some form of income off YouTube so it wouldn’t that be that big of a deal.
- pogue 8mo agoI assume it's an economics issue. As long as they continue making money off the uploads to a higher extent than it costs for storage, it works out for them.
- throw_await 8mo agoDo they make a profit nowadays
- rezonant 8mo agoLikely yes, with a margin of perhaps 38% https://news.ycombinator.com/item?id=34268536 https://news.ycombinator.com/item?id=34268536
- ranger_danger 8mo agoI wonder if anyone has ever compiled a list of channels with abnormally large numbers of videos? For example this guy has over 14,000: https://www.youtube.com/@lylehsaxon https://www.youtube.com/@lylehsaxon
- HeliumHydride 8mo agoThere is a channel with 2 million videos: https://www.youtube.com/@RoelVandePaar/videos https://www.youtube.com/@RoelVandePaar/videos One with 4 million videos: https://www.youtube.com/@NameLook https://www.youtube.com/@NameLook
- wellf 8mo agoFirst one has transcribed stack overflow to YT by the look of it
- buenzlikoder 8mo agoNameLook puts a whole new meaning to "low effort videos"
- qingcharles 8mo agoLord above. This is the worst garbage I've ever seen: https://www.youtube.com/shorts/mrOXqgShzI0 https://www.youtube.com/shorts/mrOXqgShzI0 This shit is the reason I can't afford a new HDD.
- ranger_danger 8mo agoI guess I should have mentioned I wasn't looking for automated/AI-generated videos.
- b00ty4breakfast 8mo agoAH! I've stumbled on that first fellas videos before! The videos aren't crazy complex but the sheer volume is impressive in a perverse kind of way.
- weird-eye-issue 8mo agoWhat they said only holds true because the growth continues so that the old volume of videos doesn't matter as much since there's so many more new ones each year compared to the previous year. So the question is more about whether or not it will hold true in the long term, not today
- raincole 8mo agoThe framing here is really weird. The volume of videos increasing isn't 'growth.' Videos are inventory for Youtube. They're only good when people (without adblocks!) actually watch them.
- amelius 8mo ago^ This.
- weird-eye-issue 8mo agoGrowth in this context is that there are a larger volume of videos each year. So each year a single video is exponentially a smaller and smaller percentage of the total.
- raincole 8mo agoYeah and the math doesn't check out. For example, if in year N youtube has f(N) new video. Let assume f(N) = cN^2. It's a crazy rate of growth. It's far better than the real world Youtube, which grew rather linearly. But the rate of "videos that are older than 5 years" is still faster than that, because it would be cubic instead of quadratic. Unless the it's really exponential (it isn't), "videos that are older than 5 years" will always surpass "new videos this year" eventually.
- weird-eye-issue 8mo ago> Unless the it's really exponential (it isn't), "videos that are older than 5 years" will always surpass "new videos this year" eventually. Such a weird strawman argument that you are making up. You've over thought this so much that you are missing the forest from the trees
- pwdisswordfishy 8mo ago> The volume of videos increases exponentially Source?
- arjie 8mo agoVideos do disappear, though. https://www.reddit.com/r/DataHoarder/comments/1ioz4x1/is_it_just_me_or_has_youtube_been_taking_down_a/ https://www.reddit.com/r/DataHoarder/comments/1ioz4x1/is_it_... Searching hn.algolia.com for examples will yield numerous ones. https://news.ycombinator.com/item?id=23758547 https://news.ycombinator.com/item?id=23758547 https://bsky.app/profile/sinevibes.bsky.social/post/3lhazuyn5as2t https://bsky.app/profile/sinevibes.bsky.social/post/3lhazuyn...
- Kwpolska 8mo agoOf course videos disappear for copyright, ToS violations, or when the uploaders remove them. They do not disappear just because nobody watched them.
- Gigachad 8mo agoThere’s a whole activity around discovering random 15 year old videos with almost no views. It’s usually some random home video
- epolanski 8mo agoA friend of mine worked two years in YouTube as a content admin. Basically being given videos to watch all day, especially coming from the middle east (this was ISIS time so any video from the area had someone watching it as soon as uploaded). Needless to say there's endless gold no view videos according to him. It's also interesting that it was no open secret that already in 2018 they were all told that they were essentially training machines to do their job.
- nabhasablue 8mo agoI was interested in the same thing and built a search for it https://ytstalker.mov https://ytstalker.mov
- leephillips 8mo agoThey also disappear when the government of Pakistan tells Google to erase them: https://lee-phillips.org/youtube/ https://lee-phillips.org/youtube/
- ntoskrnl_exe 8mo agoWouldn't it also be a performance nightmare? The energy bill for scanning through the terabytes of metadata would be comparable to that of several months of AI training, not to mention the time it would take. Then deleting a few million random 360p videos and putting MrBeast in their place would result in insane fragmentation of the new files. It might really just be cheaper to keep buying new HDDs.
- moffkalast 8mo agoBesides with their search deteriorating to the point where a direct video title doesn't result in a match, nobody can see those videos anyway and they don't have to cache them.
- sfn42 8mo agoIt's not just the search deteriorating. The frontend is littered with bugs. If you write a comment and try to highlight and delete part of that comment, it'll often delete the part you didn't highlight. So apparently they implemented their own textfield for some reason and also fucked it up. It's been like that for years. The youtube shorts thing is buggy as shit, it'll just stop working a lot of the time, just won't load a video. Some times you have to go back and forth a few times to get it to load. It'll often desync the comments from the video, so you're seeing comments from a different video. Some times the sound from one short plays over the visuals of another. It only checks for notifications when you open the website from a new tab, so if you want to see if you have any notifications you have to open youtube in a new tab. Refreshing doesn't work. Seems like all the competent developers have left.
- r_lee 8mo agoand if you do a hard refresh on the webapp, it literally takes like 10 seconds for the homepage to load
- sfn42 8mo agoYeah, one that I forgot to mention is if you pause a youtube short and go to a different tab, the short will unpause in the background, or it might change to an entirely different short and start playing that.
- jl6 8mo agoOne day, it will matter. Not even Google can escape the consequences of infinite growth. Kryder's Law is over. We cannot rely on storage getting cheaper faster than we can fill it, and orgs cannot rely on being able to extract more value from data than it costs to store it. Every other org knows this already. The only difference with Google is that they have used their ad cash generator to postpone their reality check moment. One day, somebody is going to be tasked with deciding what gets deleted. It won't be pretty. Old and unloved video will fade into JPEG noise as the compression ratio gets progressively cranked, until all that remains is a textual prompt designed to feed an AI model that can regenerate a facsimile of the original.
- asah 8mo agoYou can see how Google rolls with how they deleted old Gmail accounts - years of notice, lots of warnings, etc. They finally started deletions recently, and I haven't heard a whimper from anyone (yet).
- flux3125 8mo agoThe problem is that some content creators have already passed away (and others will pass away by then), and their videos will likely be deleted forever.
- dessimus 8mo agoMonuments erode away and memories of those enshrined are lost time as well, nothing lasts forever.
- bentcorner 8mo agoI met a user from an antique land Who said: Two squares of a clip of video Stand in at the end of the search. Near them, Lossly compressed, a profile with a pfp, whose smile, And vacant eyes, and shock of content baiting, Tell that its creator well those passions read Which yet survive, stamped on these unclicked things, The hand that mocked them and the heart that fed: And on the title these words appear: "My name is Ozymandias, Top Youtuber of All Time: Look on my works, ye Mighty, and like and subscribe!" No other video beside remains. Round the decay Of that empty profile, boundless and bare The lone and level page stretch far away.
- MagicMoonlight 8mo agoNow that they can harvest it all for AI training, that decision was the cheapest and greatest thing they ever did. Imagine trying to pay for all that content, nobody on earth would be able or willing to supply it.
- paulryanrogers 8mo agoPeerTube is a thing. I like to think without centralized players like YT, that P2P supported federation may have gained a better foothold.
- water9 8mo agoThere’s still time
- lysp 8mo agoI seem to recall reading that the HD variations may get removed leaving only 480p or lower for older unwatched videos. The original upload would likely still be stored, but not available for viewing.
- Nevermark 8mo agoThat would be an odd thing to do. HD is low resolution already, and 480 is noticeably worse. If they really wanted to compress, take out every other frame, and regenerate those frames with a neural decoder. But I don't know why that would be worth the effort for a stable number of low res files either.