5 ms·

I guess it makes sense. I remember once upon a time that Twitch saved every broadcast, in full, forever. That sounds kind of ridiculous these days, but then aga

I guess it makes sense. I remember once upon a time that Twitch saved every broadcast, in full, forever. That sounds kind of ridiculous these days, but then again YouTube does still does that for everyone’s streams and makes it work. Are there very different economies of scale at work here or are Google just willing to pay the extra money?

bobnamob 2y ago
I mean surely twitch(an Amazon brand) is leaning on the s3 scale econ for storage no?
- dehrmann 2y ago
  > an Amazon brand The word you're looking for is "frugality." This is a tail use case that isn't cheap to run. https://www.aboutamazon.com/about-us/leadership-principles https://www.aboutamazon.com/about-us/leadership-principles
  - ignoramous 2y ago
    > The word you're looking for is "frugality." > https://www.aboutamazon.com/about-us/leadership-principles https://www.aboutamazon.com/about-us/leadership-principles Leadership principles? Are you saying some exec / director ("leader") is looking for a P&L bump for their promo doc?
    - anon7000 2y ago
      You have to regurgitate (reframe your personal story in the context of) leadership principles in your interviews for an IC role too.
      - genewitch 2y ago
        Individual Contributor, a euphemism for "non-management"? copilot is usually pretty good with deciphering contextual acronyms. Why "IC" instead of "NM"?
        positr0n 2y ago
        Because for whatever reason IC is an acronym widely used in the tech industry (and daily in career discussions on this website) and NM is not.
    - positr0n 2y ago
      No, frugality has been an Amazon "value" from the beginning. Look up the story of Jeff Bezos famously using an old door for a desk. Or all the execs complaining about having to share a hotel room during business trips for the first couple of decades of Amazon's existence.
jahsome 2y ago
The post says this rule doesn't apply to past broadcasts. Presumably that means the rule only applies to uploaded videos. Which I did not even realize was a feature, and I've been an avid watcher and occasional broadcaster since the justin days. Edit: others have explained elsewhere VODs are auto deleted after 60 days, and then must be converted to highlights, which will be affected. I think anyone who relies on Twitch VODs as a viewer or producer is a glutton for punishment anyway. The viewing experience is dreadful if I remember correctly, enough so I just wait for a YouTube upload anyway. In my anecdotal experience, I have probably watched several thousands of hours of live content over the last decade-plus, and maybe a half dozen VODs.
- noirbot 2y ago
  Twitch VODs aren't really any worse than any other way to watch video? I'll regularly use them when I see some stream I'm interested in while I'm busy with something else, or if I jump into something halfway and want to go back and watch the beginning. What do you think is dreadful about it?
  - Dylan16807 2y ago
    They inject a stupendous amount of ads compared to live content, otherwise they're fine. It's nice to see the chat. The place where twitch really lacks isn't on vods, it's the fact that the live player can't pause or rewind at all.
    - genewitch 2y ago
      yeah, and you can't ffwd live, either. I submitted a PR but it'll take a while for them to implement
      - Dylan16807 2y ago
        I'm so used to this being a legitimate problem that I didn't even realize you were joking when I first read this comment. So let me elaborate. I find myself needing to fast forward live quite often. Even without a pause button, Twitch can fall behind by several seconds or even a minute, especially when there are connection problems in the pipeline. While it does have some ability to run a couple percent fast to catch up, often that's not enough. I have some javascript I keep around to force twitch to run at 1.3x speed for a while, while on youtube it's an easy builtin toggle to run fast and eat through the buffer until there's no buffer left.
wongarsu 2y ago
However unlike Twitch, Youtube doesn't save recordings of livestreams over 12 hours. Which means that subathons (a format where viewers extend the duration of the stream by donating money) don't get recorded on Youtube.
- unshavedyak 2y ago
  That annoys me quite a bit. I regularly watch Dota series and they can run for 10-14h regularly. It sucks to see it cut off after 12h.
  - achierius 2y ago
    Especially since that tail bit is often going to be the best part -- game 5 bo5 grand finals &c!
- genewitch 2y ago
  Okay, but, this is known, right? So the streamer needs to tell yt-dlp to copy the stream to disk so they can re-upload it right after? In fact, this is possible to do programatically probably with some minor fiddling with OBS Studio, right? is there a hosted product here? Pay-for-use video FXP?
akimbostrawman 2y ago
twitch is nowhere near the size of youtube even if streams are usually longer than videos. they also have probably not even 1% of the channel amount and at this point there are more streamer on youtube than twitch. if youtube (google) can, then twitch (amazon) should too.
mjrpes 2y ago
It's interesting what this situation would be like if HDD capacity hadn't stopped its exponential growth in 2010: https://imgur.com/a/lWdcjX7 https://imgur.com/a/lWdcjX7 We would be at a penny per terabyte of space. If AV1 in HD can store 400 hours of video per TB, the roughly 24TB to store a 24/7 stream over the course of a year would cost only 25 cents. Providers could keep all video content indefinitely. Perhaps there's some benefit to this exponential growth coming to an end. Imagine a surveillance state that had near limitless storage and could keep 24/7 recordings indefinitely of cameras on every street, house, vehicle, etc.
- fc417fc802 2y ago
  Unfortunately that remains a concern. The current research on ML based video codecs is yielding almost unbelievable size results.
  - pdimitar 2y ago
    Where can I read more?
    - fc417fc802 2y ago
      I guess this [0] might be a decent starting point. Also consider size comparisons of various gaussian splatting techniques (ex [1] & [2]). Static 3D scenes aren't technically the same thing as a 2D time series of frames, but since videos are generally composed of highly spatially related data there's bound to be overlap (ex [1] has already been extended to support dynamic scenes). And keep in mind that security footage has an awful lot of both static and repetitive objects. It's an ideal target for more complicated approaches to compression. [0] https://github.com/ppingzhang/Awesome-Deep-Learning-Based-Video-Compression https://github.com/ppingzhang/Awesome-Deep-Learning-Based-Vi... [1] https://maincold2.github.io/c3dgs/ https://maincold2.github.io/c3dgs/ [2] https://jasonlsc.github.io/nerfcodec_homepage/ https://jasonlsc.github.io/nerfcodec_homepage/
      - pdimitar 2y ago

bobnamob 2y ago

I mean surely twitch(an Amazon brand) is leaning on the s3 scale econ for storage no?

dehrmann 2y ago
> an Amazon brand The word you're looking for is "frugality." This is a tail use case that isn't cheap to run. https://www.aboutamazon.com/about-us/leadership-principles https://www.aboutamazon.com/about-us/leadership-principles
- ignoramous 2y ago
  > The word you're looking for is "frugality." > https://www.aboutamazon.com/about-us/leadership-principles https://www.aboutamazon.com/about-us/leadership-principles Leadership principles? Are you saying some exec / director ("leader") is looking for a P&L bump for their promo doc?
  - anon7000 2y ago
    You have to regurgitate (reframe your personal story in the context of) leadership principles in your interviews for an IC role too.
    - genewitch 2y ago
      Individual Contributor, a euphemism for "non-management"? copilot is usually pretty good with deciphering contextual acronyms. Why "IC" instead of "NM"?
      - positr0n 2y ago
        Because for whatever reason IC is an acronym widely used in the tech industry (and daily in career discussions on this website) and NM is not.
  - positr0n 2y ago
    No, frugality has been an Amazon "value" from the beginning. Look up the story of Jeff Bezos famously using an old door for a desk. Or all the execs complaining about having to share a hotel room during business trips for the first couple of decades of Amazon's existence.

jahsome 2y ago

The post says this rule doesn't apply to past broadcasts. Presumably that means the rule only applies to uploaded videos. Which I did not even realize was a feature, and I've been an avid watcher and occasional broadcaster since the justin days. Edit: others have explained elsewhere VODs are auto deleted after 60 days, and then must be converted to highlights, which will be affected. I think anyone who relies on Twitch VODs as a viewer or producer is a glutton for punishment anyway. The viewing experience is dreadful if I remember correctly, enough so I just wait for a YouTube upload anyway. In my anecdotal experience, I have probably watched several thousands of hours of live content over the last decade-plus, and maybe a half dozen VODs.

noirbot 2y ago
Twitch VODs aren't really any worse than any other way to watch video? I'll regularly use them when I see some stream I'm interested in while I'm busy with something else, or if I jump into something halfway and want to go back and watch the beginning. What do you think is dreadful about it?
- Dylan16807 2y ago
  They inject a stupendous amount of ads compared to live content, otherwise they're fine. It's nice to see the chat. The place where twitch really lacks isn't on vods, it's the fact that the live player can't pause or rewind at all.
  - genewitch 2y ago
    yeah, and you can't ffwd live, either. I submitted a PR but it'll take a while for them to implement
    - Dylan16807 2y ago
      I'm so used to this being a legitimate problem that I didn't even realize you were joking when I first read this comment. So let me elaborate. I find myself needing to fast forward live quite often. Even without a pause button, Twitch can fall behind by several seconds or even a minute, especially when there are connection problems in the pipeline. While it does have some ability to run a couple percent fast to catch up, often that's not enough. I have some javascript I keep around to force twitch to run at 1.3x speed for a while, while on youtube it's an easy builtin toggle to run fast and eat through the buffer until there's no buffer left.

wongarsu 2y ago

However unlike Twitch, Youtube doesn't save recordings of livestreams over 12 hours. Which means that subathons (a format where viewers extend the duration of the stream by donating money) don't get recorded on Youtube.

unshavedyak 2y ago
That annoys me quite a bit. I regularly watch Dota series and they can run for 10-14h regularly. It sucks to see it cut off after 12h.
- achierius 2y ago
  Especially since that tail bit is often going to be the best part -- game 5 bo5 grand finals &c!
genewitch 2y ago
Okay, but, this is known, right? So the streamer needs to tell yt-dlp to copy the stream to disk so they can re-upload it right after? In fact, this is possible to do programatically probably with some minor fiddling with OBS Studio, right? is there a hosted product here? Pay-for-use video FXP?

akimbostrawman 2y ago

twitch is nowhere near the size of youtube even if streams are usually longer than videos. they also have probably not even 1% of the channel amount and at this point there are more streamer on youtube than twitch. if youtube (google) can, then twitch (amazon) should too.

mjrpes 2y ago

It's interesting what this situation would be like if HDD capacity hadn't stopped its exponential growth in 2010: https://imgur.com/a/lWdcjX7 https://imgur.com/a/lWdcjX7 We would be at a penny per terabyte of space. If AV1 in HD can store 400 hours of video per TB, the roughly 24TB to store a 24/7 stream over the course of a year would cost only 25 cents. Providers could keep all video content indefinitely. Perhaps there's some benefit to this exponential growth coming to an end. Imagine a surveillance state that had near limitless storage and could keep 24/7 recordings indefinitely of cameras on every street, house, vehicle, etc.

fc417fc802 2y ago
Unfortunately that remains a concern. The current research on ML based video codecs is yielding almost unbelievable size results.
- pdimitar 2y ago
  Where can I read more?
  - fc417fc802 2y ago
    I guess this [0] might be a decent starting point. Also consider size comparisons of various gaussian splatting techniques (ex [1] & [2]). Static 3D scenes aren't technically the same thing as a 2D time series of frames, but since videos are generally composed of highly spatially related data there's bound to be overlap (ex [1] has already been extended to support dynamic scenes). And keep in mind that security footage has an awful lot of both static and repetitive objects. It's an ideal target for more complicated approaches to compression. [0] https://github.com/ppingzhang/Awesome-Deep-Learning-Based-Video-Compression https://github.com/ppingzhang/Awesome-Deep-Learning-Based-Vi... [1] https://maincold2.github.io/c3dgs/ https://maincold2.github.io/c3dgs/ [2] https://jasonlsc.github.io/nerfcodec_homepage/ https://jasonlsc.github.io/nerfcodec_homepage/
    - pdimitar 2y ago