3 ms·

What does this mean? That websites could connect directly to an FFmpeg instance and receive an audio- and/or video-stream? Phoronix has a somewhat more informa

What does this mean? That websites could connect directly to an FFmpeg instance and receive an audio- and/or video-stream? Phoronix has a somewhat more informative page: https://www.phoronix.com/news/FFmpeg-Lands-WHIP-Muxer https://www.phoronix.com/news/FFmpeg-Lands-WHIP-Muxer

bigfishrunning 1y ago
It means that programs that use the FFmpeg libraries (looks like libavformat specifically) can consume webrtc streams
- qwertox 1y ago
  So it's only the receiving part of WebRTC, now being able to use WHIP in order to ask a server for a stream?
  - Sean-Der 1y ago
    Currently only the sending part! WHIP is 'pushing media via WebRTC' - https://www.ietf.org/archive/id/draft-ietf-wish-whip-01.html https://www.ietf.org/archive/id/draft-ietf-wish-whip-01.html WHEP is 'pulling media via WebRTC' - https://github.com/wish-wg/webrtc-http-egress-protocol/blob/main/draft-ietf-wish-whep.md https://github.com/wish-wg/webrtc-http-egress-protocol/blob/... WHEP isn't standardized/still changing a bit. After it lands I will try and get it into OBS/FFmpeg (and more)
    - jcelerier 1y ago
      Hmm, what does that mean for instance for workloads that use gstreamer's whepsrc? Is there a risk of incompatibility of a whep server running today with next year webrtc?
- okdood64 1y ago
  I still don't understand any practical use cases. Can you give some examples? (I'm not being obtuse here I'm genuinely curious what this can enable now.)
  - ninkendo 1y ago
    My guess is you could more easily build an open source client for whatever video conferencing system you want that uses WebRTC (most services like teams, discord, zoom, etc seem to use WebRTC as a fallback for browsers, if not using it wholesale for everything, although there may be countermeasures to block unofficial clients.)
  - lmm 1y ago
    My first thought is a nice way to save a stream in whatever format you want (e.g. transcode for watching on an old phone or something on your commute), just ffmpeg -i <stream> and then all your usual video format options, instead of having to download it and then convert it afterwards. ffmpeg also has some processing abilities of its own, so you could e.g. greenscreen (chroma key) from a stream onto an existing video background. ffmpeg is a pretty low-level building block and as others have said, it's mostly used as a library - a lot of video players or processing tools can now add support for stream inputs easily, and that's probably where the biggest impact is.
  - jcelerier 1y ago
    I want my desktop app https://ossia.io https://ossia.io which uses ffmpeg to be able to send & receive video to another computer over internet without having to fiddle with opening ports on each other's routers. This combined with a server like vdo.ninja solves that.
  - darkvertex 1y ago
    WebRTC excels at sub-second latency peer to peer, so you can do near-realtime video, so anywhere that is useful. Say you wanted to do a virtual portal installation connecting views from two different cities with live audio, you could have ffmpeg feed off a professional cinema or DSLR camera device with a clean audio feed and stream that over WebRTC into a webpage-based live viewer. Or say you wanna do a webpage that remote controls a drone or rover robot, it would be great for that.
    - Sesse__ 1y ago
      The irony is that you don't _actually_ need WebRTC to get subsecond latency; you can fairly reliably get ~100–200ms (plus network latency) with a completely normal TCP stream.[1] But since browsers have effectively standardized on HLS, whose design is completely antithetical to low-latency (you _can_ do low-latency HLS, but only with heroic efforts), low-latency streaming video has never really been part of their bread and butter. So instead, we abuse a _much_ more complicated protocol (WebRTC), because that happens to hit a path that was meant for low-latency videoconferencing. (I did sub-100ms glass-to-glass streaming with VLC back in the day, so it is eminently possible. But the browser is in your way.) [1] Much less than that is going to be tricky under non-perfect network conditions, because once you start having any sort of packet drop, you want to go from TCP's retransmission regime and instead start dropping packets, take the artifacts for a little while, and then go on.
- dark-star 1y ago
  Are there any popular/well-known WebRTC senders (or servers)? I'm pretty sure this is not for YouTube etc., right? So what would I watch through WebRTC?
  - Sean-Der 1y ago
    Twitch supports WHIP today. Lots of WebRTC services support WHIP (Cloudflare, LiveKit, Dolby...) webrtcHacks has an article on it[0] kind of old, but captures the spirit of it! [0] https://webrtchacks.com/tag/simulcast/ https://webrtchacks.com/tag/simulcast/

bigfishrunning 1y ago

It means that programs that use the FFmpeg libraries (looks like libavformat specifically) can consume webrtc streams

qwertox 1y ago
So it's only the receiving part of WebRTC, now being able to use WHIP in order to ask a server for a stream?
- Sean-Der 1y ago
  Currently only the sending part! WHIP is 'pushing media via WebRTC' - https://www.ietf.org/archive/id/draft-ietf-wish-whip-01.html https://www.ietf.org/archive/id/draft-ietf-wish-whip-01.html WHEP is 'pulling media via WebRTC' - https://github.com/wish-wg/webrtc-http-egress-protocol/blob/main/draft-ietf-wish-whep.md https://github.com/wish-wg/webrtc-http-egress-protocol/blob/... WHEP isn't standardized/still changing a bit. After it lands I will try and get it into OBS/FFmpeg (and more)
  - jcelerier 1y ago
    Hmm, what does that mean for instance for workloads that use gstreamer's whepsrc? Is there a risk of incompatibility of a whep server running today with next year webrtc?
okdood64 1y ago
I still don't understand any practical use cases. Can you give some examples? (I'm not being obtuse here I'm genuinely curious what this can enable now.)
- ninkendo 1y ago
  My guess is you could more easily build an open source client for whatever video conferencing system you want that uses WebRTC (most services like teams, discord, zoom, etc seem to use WebRTC as a fallback for browsers, if not using it wholesale for everything, although there may be countermeasures to block unofficial clients.)
- lmm 1y ago
  My first thought is a nice way to save a stream in whatever format you want (e.g. transcode for watching on an old phone or something on your commute), just ffmpeg -i <stream> and then all your usual video format options, instead of having to download it and then convert it afterwards. ffmpeg also has some processing abilities of its own, so you could e.g. greenscreen (chroma key) from a stream onto an existing video background. ffmpeg is a pretty low-level building block and as others have said, it's mostly used as a library - a lot of video players or processing tools can now add support for stream inputs easily, and that's probably where the biggest impact is.
- jcelerier 1y ago
  I want my desktop app https://ossia.io https://ossia.io which uses ffmpeg to be able to send & receive video to another computer over internet without having to fiddle with opening ports on each other's routers. This combined with a server like vdo.ninja solves that.
- darkvertex 1y ago
  WebRTC excels at sub-second latency peer to peer, so you can do near-realtime video, so anywhere that is useful. Say you wanted to do a virtual portal installation connecting views from two different cities with live audio, you could have ffmpeg feed off a professional cinema or DSLR camera device with a clean audio feed and stream that over WebRTC into a webpage-based live viewer. Or say you wanna do a webpage that remote controls a drone or rover robot, it would be great for that.
  - Sesse__ 1y ago
    The irony is that you don't _actually_ need WebRTC to get subsecond latency; you can fairly reliably get ~100–200ms (plus network latency) with a completely normal TCP stream.[1] But since browsers have effectively standardized on HLS, whose design is completely antithetical to low-latency (you _can_ do low-latency HLS, but only with heroic efforts), low-latency streaming video has never really been part of their bread and butter. So instead, we abuse a _much_ more complicated protocol (WebRTC), because that happens to hit a path that was meant for low-latency videoconferencing. (I did sub-100ms glass-to-glass streaming with VLC back in the day, so it is eminently possible. But the browser is in your way.) [1] Much less than that is going to be tricky under non-perfect network conditions, because once you start having any sort of packet drop, you want to go from TCP's retransmission regime and instead start dropping packets, take the artifacts for a little while, and then go on.
dark-star 1y ago
Are there any popular/well-known WebRTC senders (or servers)? I'm pretty sure this is not for YouTube etc., right? So what would I watch through WebRTC?
- Sean-Der 1y ago
  Twitch supports WHIP today. Lots of WebRTC services support WHIP (Cloudflare, LiveKit, Dolby...) webrtcHacks has an article on it[0] kind of old, but captures the spirit of it! [0] https://webrtchacks.com/tag/simulcast/ https://webrtchacks.com/tag/simulcast/