7 ms·

Stanford Researchers Launch Free TV Service to Improve Live Streaming Using AI

114 points by fomopop 8y ago

sand500 8y ago
This is a pretty cool idea Note, you still have to deal with TV ads and there is no scrubbing. I wonder if you could make your own client to this and get DVR functionality? Actually since they are just grabbing over-the-air signals, I wonder if there is a SDR project for this.
daveevad 8y ago
Anyone with legal knowledge about how services like this and https://www.locast.org/ https://www.locast.org/ are able to operate? It doesn't seem significantly different to me than the aereo tv situation. Very cool project!
- fomopop 8y ago
  In the case of Locast, part of the Aereo ruling was that it was a for profit service. Locast is able to do this because they are not charging the end user.
- dest 8y ago
  IANAL but in Europe, for such streams to be legal you need to use them in reach of the Hertzian signals of TVs. See about VCast for example. https://www.nextinpact.com/news/105549-lextension-copie-privee-dans-cloud-entre-mains-justice-europeenne.htm https://www.nextinpact.com/news/105549-lextension-copie-priv...
voltagex_ 8y ago
Edit: bitrate and resolution varies - I've seen bitrates as high as 20 megabit/s.
voltagex_ 8y ago
It's quite nice, but it's definitely going to get a DRM layer and geoblocking applied to it at some stage.
- wmf 8y ago
  More likely they'll never expand beyond 500 "experimental" accounts and shut it down once their research is finished. I don't intend this as a criticism of the researchers; the more academic and experimental you make something look the less likely you will be threatened/sued into compliance.
keithwinstein 8y ago
Hi folks -- study PI here. Happy to answer any questions. We're working on a paper for submission to a research conference soon, so every viewer gives us helpful data for the system to learn in-situ and improve its performance over different kinds of Internet connections and congestion situations. (We are randomizing connections to get either our in-development algorithm or a bunch of competing ABR and congestion-control schemes so we can properly evaluate, so if you get a stall or bad quality...... we'll blame that.) All the source code is at https://github.com/StanfordSNR/puffer https://github.com/StanfordSNR/puffer and is or will be released with an open-source license. The study is led by my doctoral student Francis Yan, along with Sadjad Fouladi, Hudson Ayers, Chenzhi Zhu, and my colleague Philip Levis. The goal is to train video-streaming algorithms (bitrate selection and congestion control) online, continually, in situ, to minimize stalls and deliver the best picture quality across a diverse range of real-world Internet connections and users. This is from some of the same people who brought you Salsify (functional video compression for lower-latency real-time video: https://snr.stanford.edu/salsify https://snr.stanford.edu/salsify), Lepton (distributed/parallel JPEG compression: https://github.com/dropbox/lepton https://github.com/dropbox/lepton), Mosh (functional terminal emulation for mobility: https://mosh.org https://mosh.org), Remy (machine learning for congestion control; http://mit.edu/remy http://mit.edu/remy), and the Pantheon of Congestion Control (https://pantheon.stanford.edu https://pantheon.stanford.edu). There are some unusual details of the Puffer system to make experimentation easier (and, we hope, performance better): the ABR algorithm is server-side, and video is streamed continuously over a WebSocket and the client gives asynchronous feedback, instead of using DASH HTTP request/replies. The tcp_info struct (including the delivery rate estimate) from the congestion-control layer is plumbed through to the ABR, and to the neural network that tries to predict "how long will it take to send a video chunk of length y," so it's weakly cross-layer. The "transmission-time predictor" is probabilistic instead of simply producing a point estimate. We encode with libx264 and then measure the SSIM of every encoded chunk to calculate a reward for each possible stream (instead of making the assumption that more bitrate is better -- the correlation is not so good when you consider different times in the same video). And we encode 10 variants of each channel, so more than typical. And everything is done at 60 fps. Of course some of these details mean these algorithms are not going to be deployed any time soon on a production CDN or streaming service, but the hope is to demonstrate the value of some of the ideas in an academic setting for future commercial deployment. The use of WebSockets, MSE, and Opus is why this doesn't work on Safari or on iOS. FAQ here: https://puffer.stanford.edu/faq/ https://puffer.stanford.edu/faq/ Also I'm happy we were able to make some of the Grafana monitoring public: https://puffer.stanford.edu/monitoring https://puffer.stanford.edu/monitoring
abhiminator 8y ago
Streaming it in another browser window as a part of my Mechanical Turk HIT assignment as I type, and I have to say -- I'm blown away by the incredible picture quality and buffer-free performance. Puffer team has done a great job. As a sidenote, I just realized how annoying cable TV ads are. The content to advertising ratio on cable is insanely bad. This 'Puffer' experience just validated my decision to stay away from cable TV.
- ehsankia 8y ago
  Switching channel takes less than a quarter of a second, this is pretty impressive.
  - ian0 8y ago
    Indeed - it's very well done. I think what makes it so smooth is something quite simple. Its letting the previous stream run while it buffers the new one and switches you over?
    - rasz 8y ago
      That would still make you wait for the switch. Switching channels on digital TV is slow due to key frames - you need to wait until one full Intra-frame of the new channel has been decoded. Here client can simply request Intra-frame on demand and receive one immediately.
    - ehsankia 8y ago
      Higher up a comment mentions they use websocket. I believe it uses the same connection, and in the backend it swaps you to a different feed, allowing it to be very quick. So really you only get one keyframe worth of black before the new video stream starts.
milankragujevic 8y ago
I'm going to be experimenting with the source code, I really need some good basis for live streaming TV over the internet (OTT). Unrelated, but important for me, TV stations in my country (Serbia), forbid rebroadcasting but they are available OTA, with a DVB-T2 receiver and an aerial/antenna. However, some cable providers cannot show the programs, and if they do show them, they have negotiated special licenses for that. How do I make a case that by being OTA Free, and not DRM'd, I, by not changing the contents of the broadcast, should be allowed to rebroadcast to people for no financial gain to myself or the TV stations (other than the ads the TV stations put inside of the broadcast content) (all to serve as an experiment)?
masonic 8y ago
Is closed captioning supported?
phillipseamore 8y ago
This is very impressive. FYI: Delaying the buffer spinner, not showing it unless buffering has been (let's say) >300ms for instance, makes the switching feel a lot faster. If you could also get rid of the black between switching out the media source buffer, they would feel instant.
bane 8y ago
This is actually faster at channel switching than my home fiber TV service.
compsciphd 8y ago
wondering what the odds are they get a takedown from the NFL during the super bowl or preemptively won't show CBS during that time period?
homero 8y ago
The quality and speed is incredible

114 points by fomopop 8y ago

This is a pretty cool idea Note, you still have to deal with TV ads and there is no scrubbing. I wonder if you could make your own client to this and get DVR functionality? Actually since they are just grabbing over-the-air signals, I wonder if there is a SDR project for this.

daveevad 8y ago

Anyone with legal knowledge about how services like this and https://www.locast.org/ https://www.locast.org/ are able to operate? It doesn't seem significantly different to me than the aereo tv situation. Very cool project!

fomopop 8y ago
In the case of Locast, part of the Aereo ruling was that it was a for profit service. Locast is able to do this because they are not charging the end user.
dest 8y ago
IANAL but in Europe, for such streams to be legal you need to use them in reach of the Hertzian signals of TVs. See about VCast for example. https://www.nextinpact.com/news/105549-lextension-copie-privee-dans-cloud-entre-mains-justice-europeenne.htm https://www.nextinpact.com/news/105549-lextension-copie-priv...

voltagex_ 8y ago

Edit: bitrate and resolution varies - I've seen bitrates as high as 20 megabit/s.

voltagex_ 8y ago

It's quite nice, but it's definitely going to get a DRM layer and geoblocking applied to it at some stage.

wmf 8y ago
More likely they'll never expand beyond 500 "experimental" accounts and shut it down once their research is finished. I don't intend this as a criticism of the researchers; the more academic and experimental you make something look the less likely you will be threatened/sued into compliance.

keithwinstein 8y ago

Hi folks -- study PI here. Happy to answer any questions. We're working on a paper for submission to a research conference soon, so every viewer gives us helpful data for the system to learn in-situ and improve its performance over different kinds of Internet connections and congestion situations. (We are randomizing connections to get either our in-development algorithm or a bunch of competing ABR and congestion-control schemes so we can properly evaluate, so if you get a stall or bad quality...... we'll blame that.) All the source code is at https://github.com/StanfordSNR/puffer https://github.com/StanfordSNR/puffer and is or will be released with an open-source license. The study is led by my doctoral student Francis Yan, along with Sadjad Fouladi, Hudson Ayers, Chenzhi Zhu, and my colleague Philip Levis. The goal is to train video-streaming algorithms (bitrate selection and congestion control) online, continually, in situ, to minimize stalls and deliver the best picture quality across a diverse range of real-world Internet connections and users. This is from some of the same people who brought you Salsify (functional video compression for lower-latency real-time video: https://snr.stanford.edu/salsify https://snr.stanford.edu/salsify), Lepton (distributed/parallel JPEG compression: https://github.com/dropbox/lepton https://github.com/dropbox/lepton), Mosh (functional terminal emulation for mobility: https://mosh.org https://mosh.org), Remy (machine learning for congestion control; http://mit.edu/remy http://mit.edu/remy), and the Pantheon of Congestion Control (https://pantheon.stanford.edu https://pantheon.stanford.edu). There are some unusual details of the Puffer system to make experimentation easier (and, we hope, performance better): the ABR algorithm is server-side, and video is streamed continuously over a WebSocket and the client gives asynchronous feedback, instead of using DASH HTTP request/replies. The tcp_info struct (including the delivery rate estimate) from the congestion-control layer is plumbed through to the ABR, and to the neural network that tries to predict "how long will it take to send a video chunk of length y," so it's weakly cross-layer. The "transmission-time predictor" is probabilistic instead of simply producing a point estimate. We encode with libx264 and then measure the SSIM of every encoded chunk to calculate a reward for each possible stream (instead of making the assumption that more bitrate is better -- the correlation is not so good when you consider different times in the same video). And we encode 10 variants of each channel, so more than typical. And everything is done at 60 fps. Of course some of these details mean these algorithms are not going to be deployed any time soon on a production CDN or streaming service, but the hope is to demonstrate the value of some of the ideas in an academic setting for future commercial deployment. The use of WebSockets, MSE, and Opus is why this doesn't work on Safari or on iOS. FAQ here: https://puffer.stanford.edu/faq/ https://puffer.stanford.edu/faq/ Also I'm happy we were able to make some of the Grafana monitoring public: https://puffer.stanford.edu/monitoring https://puffer.stanford.edu/monitoring

abhiminator 8y ago

Streaming it in another browser window as a part of my Mechanical Turk HIT assignment as I type, and I have to say -- I'm blown away by the incredible picture quality and buffer-free performance. Puffer team has done a great job. As a sidenote, I just realized how annoying cable TV ads are. The content to advertising ratio on cable is insanely bad. This 'Puffer' experience just validated my decision to stay away from cable TV.

ehsankia 8y ago
Switching channel takes less than a quarter of a second, this is pretty impressive.
- ian0 8y ago
  Indeed - it's very well done. I think what makes it so smooth is something quite simple. Its letting the previous stream run while it buffers the new one and switches you over?
  - rasz 8y ago
    That would still make you wait for the switch. Switching channels on digital TV is slow due to key frames - you need to wait until one full Intra-frame of the new channel has been decoded. Here client can simply request Intra-frame on demand and receive one immediately.
  - ehsankia 8y ago
    Higher up a comment mentions they use websocket. I believe it uses the same connection, and in the backend it swaps you to a different feed, allowing it to be very quick. So really you only get one keyframe worth of black before the new video stream starts.

milankragujevic 8y ago

I'm going to be experimenting with the source code, I really need some good basis for live streaming TV over the internet (OTT). Unrelated, but important for me, TV stations in my country (Serbia), forbid rebroadcasting but they are available OTA, with a DVB-T2 receiver and an aerial/antenna. However, some cable providers cannot show the programs, and if they do show them, they have negotiated special licenses for that. How do I make a case that by being OTA Free, and not DRM'd, I, by not changing the contents of the broadcast, should be allowed to rebroadcast to people for no financial gain to myself or the TV stations (other than the ads the TV stations put inside of the broadcast content) (all to serve as an experiment)?

masonic 8y ago

Is closed captioning supported?

phillipseamore 8y ago

This is very impressive. FYI: Delaying the buffer spinner, not showing it unless buffering has been (let's say) >300ms for instance, makes the switching feel a lot faster. If you could also get rid of the black between switching out the media source buffer, they would feel instant.

bane 8y ago

This is actually faster at channel switching than my home fiber TV service.

compsciphd 8y ago

wondering what the odds are they get a takedown from the NFL during the super bowl or preemptively won't show CBS during that time period?

homero 8y ago

The quality and speed is incredible