5 ms·

You don’t need to keep everything “hot” all the time. Storage tiers exist for a reason.

by k8sToGo 2y ago

grepfru_it 2y ago
My 15 year old videos with 30 views still load nearly instantly. It’s as close to hot as hot is
- SirMaster 2y ago
  But they are tiny 480p files yeah?
- crazygringo 2y ago
  Speed/latency doesn't tell you much, because it's all on a hard drive somewhere. The question is whether YT is serving up the one (redundantly-backed storage) copy they have of your almost-never-watched video, or whether it's serving it up from one of 1,000+ copies it's made across the globe for currently popular videos.
- NBJack 2y ago
  Think about what you actually need to start a video. Maybe a dozen MB? After that, you can plunge into colder storages and warm things up as you stream. Additionally, if you need longer to 'defrost' things, just cache a few more MB at the front. Cheat a bit by assuming 480p to start with if you need to; even less to store.
  - londons_explore 2y ago
    There is also location location location. Maybe Google holds your content in 7 data centers round the world (~1 per continent for planned maintenance + latency + reduced oceanic fiber usage). But with old rarely streamed content they might cut that down to just 3.
- spatulon 2y ago
  That doesn't match my experience. I have some unlisted videos that I or a small handful of friends might go back and watch once a year, and it takes several seconds of loading before they start playing. It's very noticeably different from the near-instant loading of most videos I watch.
pinakinathc 2y ago
Exactly! This was my go-to approach for reducing storage costs. Customers don't get spooked when they get an extra 1s delay for something they search once in a month. However, an extra 30ms delay in "everyday content" is a sure way to loose your users. However, implementing this in practice is non-trivial. Knowing what is "everyday content" versus what is "once a month content". To add more complexity -- you have these semi-predictable hype-waves especially two peaks in case of most YT videos where a "once-a-month" content becomes an "everyday" content before again becoming a "once-a-month" content. It feels you could specifically optimise for this -- reduce storage costs without sacrificing UX.
- c0balt 2y ago
  > To add more complexity -- you have these semi-predictable hype-waves especially two peaks in case of most YT videos where a "once-a-month" content becomes an "everyday" content before again becoming a "once-a-month" content. Caching is hard but this sounds like an ARC would likely catch this, if it occurs on a small number of videos concurrently.

You don’t need to keep everything “hot” all the time. Storage tiers exist for a reason.

grepfru_it 2y ago

My 15 year old videos with 30 views still load nearly instantly. It’s as close to hot as hot is

SirMaster 2y ago
But they are tiny 480p files yeah?
crazygringo 2y ago
Speed/latency doesn't tell you much, because it's all on a hard drive somewhere. The question is whether YT is serving up the one (redundantly-backed storage) copy they have of your almost-never-watched video, or whether it's serving it up from one of 1,000+ copies it's made across the globe for currently popular videos.
NBJack 2y ago
Think about what you actually need to start a video. Maybe a dozen MB? After that, you can plunge into colder storages and warm things up as you stream. Additionally, if you need longer to 'defrost' things, just cache a few more MB at the front. Cheat a bit by assuming 480p to start with if you need to; even less to store.
- londons_explore 2y ago
  There is also location location location. Maybe Google holds your content in 7 data centers round the world (~1 per continent for planned maintenance + latency + reduced oceanic fiber usage). But with old rarely streamed content they might cut that down to just 3.
spatulon 2y ago
That doesn't match my experience. I have some unlisted videos that I or a small handful of friends might go back and watch once a year, and it takes several seconds of loading before they start playing. It's very noticeably different from the near-instant loading of most videos I watch.

pinakinathc 2y ago

Exactly! This was my go-to approach for reducing storage costs. Customers don't get spooked when they get an extra 1s delay for something they search once in a month. However, an extra 30ms delay in "everyday content" is a sure way to loose your users. However, implementing this in practice is non-trivial. Knowing what is "everyday content" versus what is "once a month content". To add more complexity -- you have these semi-predictable hype-waves especially two peaks in case of most YT videos where a "once-a-month" content becomes an "everyday" content before again becoming a "once-a-month" content. It feels you could specifically optimise for this -- reduce storage costs without sacrificing UX.

c0balt 2y ago
> To add more complexity -- you have these semi-predictable hype-waves especially two peaks in case of most YT videos where a "once-a-month" content becomes an "everyday" content before again becoming a "once-a-month" content. Caching is hard but this sounds like an ARC would likely catch this, if it occurs on a small number of videos concurrently.