6 ms·

I am heavily conflicted on this. I am happy that people get access to better codecs. Better compression and those that aren't encumbered with patents/licensing

I am heavily conflicted on this. I am happy that people get access to better codecs. Better compression and those that aren't encumbered with patents/licensing (AV1) I am sad that we are extending RTMP instead of adopting something better. I want things like * Codec negotiation happening at connect time, so we don't hardcode things * Multi-track. So users can upload multiple audio tracks, or do Simulcast. I would love a world where users upload the multiple quality levels. Then anyone could run a stream service, not just those that can afford transcodes. * P2P. Stream your OBS output directly to another user. It is frustrating to see users struggle with setting up RTMP servers/HLS when they want to share their video with just one or two other people. The other use case I really care about is co-streaming. * Standardized. I would like to see whatever protocol we use be discussed in the IETF. It frustrates me that it is just companies behind closed doors deciding things. If an individual/open source/startup has an idea or use case it is never going to make it into RTMP. I have been trying to add WebRTC support https://github.com/obsproject/obs-studio/pull/7926 https://github.com/obsproject/obs-studio/pull/7926 so I am hopeful for the future.

der_rod 4y ago
Well there just isn't "something better". Yet. MoQ (Media over QUIC) development is underway at the IETF - with involvement from Twitch/YouTube/Facebook/Cisco/etc. - but will probably still take quite some time before its finalised and can be adopted.
- Sean-Der 4y ago
  I use WebRTC today and am happy with it! Works great for broadcasting from Web, Mobile and tools that support it (GStreamer) I have a PR open to OBS for it right now https://github.com/obsproject/obs-studio/pull/7926 https://github.com/obsproject/obs-studio/pull/7926
  - tracker1 4y ago
    How many of those users are direct vs. relaying through a TURN server? I'm genuinely curious, as I kind of anticipate that the majority of users are relaying, but I guess if p2p requests can open router ports in the common config, that could work.
    - londons_explore 4y ago
      IPv6 users usually manage a direct connection, and that is ~half the world now. IPv4 users seem to manage a connection ~70% of the time in my experience. Remember only one peer needs to have a router with a suitable config for everything to work.
      - tracker1 4y ago
        I realize most of that... just didn't realize how many work in terms of direct connection, which is nice to hear.
  - londons_explore 4y ago
    When a PR sits for many months and has hundreds of comments, it probably means it's getting near to bikeshed territory and it would have been better to get small PR's in adding the groundwork and a project plan agreed with leads beforehand...
    - Sean-Der 4y ago
      Project leads created the original PR + RFC * PR https://github.com/obsproject/obs-studio/pull/7192 https://github.com/obsproject/obs-studio/pull/7192 * RFC https://github.com/obsproject/rfcs/pull/43 https://github.com/obsproject/rfcs/pull/43 I tried to keep the size as small as I could. It is just basic audio+video (no Simulcast/ICE Renegotiation etc...) It has been a tough process, but I am going to keep working on it until I can get consensus.
- Sesse__ 4y ago
  SRT is definitely “something better”. It doesn't support codec negotiation, but it's more generally much more reasonable than RTMP for video ingest.
  - der_rod 4y ago
    Doesn't support AV1.
    - Dylan16807 4y ago
      1. It looks like RTMP didn't support it, and arguably still doesn't, since they link to an "enhanced RTMP" spec. 2. "Unlike some other protocols that only support specific video and audio formats, SRT does not limit you to a specific container or codec, since it is media or content agnostic. SRT operates at the network transport level, acting as a wrapper around your content. This means it can transport any type of codec, resolution or frame rate. This is important because it can future proof workflows by working transparently with MPEG-2, H.264, and HEVC for example."
      - der_rod 4y ago
        1. Yes, it's an extension to the spec published by an organisation Adobe and Google are a part of. It extends the protocol in a backwards-compatible way. 2. SRT, in practice, is only ever used with MPEG-TS as the container for audio/video data, which does not support AV1.
        Dylan16807 4y ago
        The point is, you don't even need to extend the protocol for SRT. You just need to set up servers that are ready, just like companies are in the middle of doing for other techs. Which I'm sure is still significant work, but is it much harder to do with SRT?
        der_rod 4y ago
        > You just need to set up servers that are ready The "just" is doing a lot of work here. Extending existing RTMP infrastructure is obviously much easier than spinning up an entirely different one. And you'd still have specify a way to use anything but MPEG-TS to send audio/video over SRT, which de facto does not exist (at least not that I can find). I agree that extending RTMP isn't ideal, but other solutions have their own set of limitations, problems, and challenges. Whether that's SRT, RIST, or WebRTC. For better and worse, RTMP and dealing with it at scale is well understood.
ofchnofc 4y ago
I don't normally write this type of comment, but you are an absolute powerhouse and I'm so glad to see you pushing WebRTC, WebRTC+OBS, and playing the social game a bit to drum up support. WebRTC has so much potential, I have big ideas for using it in a number of projects, if I can ever just clone myself a couple of times. Thanks for your work and enthusiasm! I constantly dream of a much more simple, but also powerful OBS replacement, with separated components for sending inputs to a "compositing" server, that can then rebroadcast via WebRTC. Maybe with customizable templates that could be more powerful than OBS's scenes. With the ability to let viewers customize/change their views further... It's a bit much to type out all here, but the WebRTC-powered future is very exciting.
toast0 4y ago
> I would love a world where users upload the multiple quality levels. Then anyone could run a stream service, not just those that can afford transcodes. I believe both AV1 and HEVC/h.265 specify optional Scalable Video Coding, although I don't know if that's a common feature of encoders or if this integration supports it. If the stars align, it's straightforward to choose parts of the stream to throw away to fit into available bandwidth or decode capacity without transcoding. Of course, decoder support is also needed.
rcme 4y ago
Does codec negotiation really make sense? Encoding video is so expensive that you can only realistically support one or two codecs at a given time. Anything else is wasteful. > * P2P. Stream your OBS output directly to another user. It is frustrating to see users struggle with setting up RTMP servers/HLS when they want to share their video with just one or two other people. The other use case I really care about is co-streaming. The nginx RTMP plugin supports converting an RTMP stream into an HLS stream. It's pretty easy to set up. Even if OBS used WebRTC or some other p2p solution, you would still need some type of signaling / connection negotiation setup.
- Sean-Der 4y ago
  > Does codec negotiation really make sense On startup you could be connecting two clients that have completely different sets of codecs. What if you have a low powered device that only has a H264 hardware decoder? What if you have two desktops that prefer AV1. WebRTC lets each side publish what codecs they support, and their priority > It's pretty easy to set up Yea it is not bad if you are comfortable with those things. I want to empower users who have no idea what Linux, codecs or systemd is. I want to create something like 12 year old me would have used. I could have had a lot of fun with my friends if we could have done P2P. > you would still need some type of signaling / connection negotiation setup. I use QR codes for this. For public signaling is also cheap enough that it could be provided for free. I think the project name was simplewebrtc? Hosting RTMP/HLS for free is prohibitively expensive though.
  - favsq 4y ago
    >What if you have a low powered device that only has a H264 hardware decoder? What if you have two desktops that prefer AV1. You recode on the server.
    - Sean-Der 4y ago
      You don't have a server (and in my cases you don't want) * I don't want to upload my video from my security camera to a remote host. It has security and privacy implications * I don't have the bandwidth available * I don't want to pay the bandwidth and compute costs. * I don't want to incur the extra latency
MawKKe 4y ago
I've been wondering about that multi track thing w.r.t video live streams Wouldn't it make sense to have separate channels for distinct content streams (e.g game capture, face camera, overlays..) so that you could independently choose/optimize each channel's compression and thus make the whole stream more efficient? I don't know anything about compression algorithms, but I hypothesize that maybe now the algos would also be able to do a better job since the frames have less abrupt discontinuities (in the viewport perspective). This would also enable stream viewers's to activate/deactivate channels when needed. For example, when the overlay/webcam/etc occludes the view on layer (channel) below.
- kevincox 4y ago
  > so that you could independently choose/optimize each channel's compression and thus make the whole stream more efficient? In general the compression algorithm will make an intelligent decision on where to spend bits. However you may gain a bit more accuracy here. The better win may be to set priorities (maybe make the scrolling text crisp right away because it doesn't move often anyways) but I am still skeptical that you would see much benefit. > enable stream viewers's to activate/deactivate channels when needed On the other hand this would be very cool. I can also see moving away from fixed aspect ratios to moving the bits around the screen. Maybe the webcam can be above the (non-video) chat rather than over top of the video. Or even just mapping the different streams (that I have selected) into my aspect ratio better (like Google Meet and other video conferencing tools do).
zamadatix 4y ago
Even if WebRTC or something that supports all of these is added I still wouldn't be sad we are extending RTMP. If the next hot new thing requires artificially limiting the old thing then it's not really the next hot new thing.
- Sean-Der 4y ago
  The lack of collective 'internet user' ownership makes me sad. Many people want to improve RTMP, but an open discussion doesn't exist. If RTMP was in the IETF my comment wouldn't apply anymore.
  - zamadatix 4y ago
    While not specifically the IETF isn't this spec enhancement done by folks in e.g. the obs, ffmpeg, xsplit, and vlc projects? I.e. is this an ideological concern of which standards body owns it or a practical one of people not being able to improve the standard.

der_rod 4y ago

Well there just isn't "something better". Yet. MoQ (Media over QUIC) development is underway at the IETF - with involvement from Twitch/YouTube/Facebook/Cisco/etc. - but will probably still take quite some time before its finalised and can be adopted.

Sean-Der 4y ago
I use WebRTC today and am happy with it! Works great for broadcasting from Web, Mobile and tools that support it (GStreamer) I have a PR open to OBS for it right now https://github.com/obsproject/obs-studio/pull/7926 https://github.com/obsproject/obs-studio/pull/7926
- tracker1 4y ago
  How many of those users are direct vs. relaying through a TURN server? I'm genuinely curious, as I kind of anticipate that the majority of users are relaying, but I guess if p2p requests can open router ports in the common config, that could work.
  - londons_explore 4y ago
    IPv6 users usually manage a direct connection, and that is ~half the world now. IPv4 users seem to manage a connection ~70% of the time in my experience. Remember only one peer needs to have a router with a suitable config for everything to work.
    - tracker1 4y ago
      I realize most of that... just didn't realize how many work in terms of direct connection, which is nice to hear.
- londons_explore 4y ago
  When a PR sits for many months and has hundreds of comments, it probably means it's getting near to bikeshed territory and it would have been better to get small PR's in adding the groundwork and a project plan agreed with leads beforehand...
  - Sean-Der 4y ago
    Project leads created the original PR + RFC * PR https://github.com/obsproject/obs-studio/pull/7192 https://github.com/obsproject/obs-studio/pull/7192 * RFC https://github.com/obsproject/rfcs/pull/43 https://github.com/obsproject/rfcs/pull/43 I tried to keep the size as small as I could. It is just basic audio+video (no Simulcast/ICE Renegotiation etc...) It has been a tough process, but I am going to keep working on it until I can get consensus.
Sesse__ 4y ago
SRT is definitely “something better”. It doesn't support codec negotiation, but it's more generally much more reasonable than RTMP for video ingest.
- der_rod 4y ago
  Doesn't support AV1.
  - Dylan16807 4y ago
    1. It looks like RTMP didn't support it, and arguably still doesn't, since they link to an "enhanced RTMP" spec. 2. "Unlike some other protocols that only support specific video and audio formats, SRT does not limit you to a specific container or codec, since it is media or content agnostic. SRT operates at the network transport level, acting as a wrapper around your content. This means it can transport any type of codec, resolution or frame rate. This is important because it can future proof workflows by working transparently with MPEG-2, H.264, and HEVC for example."
    - der_rod 4y ago
      1. Yes, it's an extension to the spec published by an organisation Adobe and Google are a part of. It extends the protocol in a backwards-compatible way. 2. SRT, in practice, is only ever used with MPEG-TS as the container for audio/video data, which does not support AV1.
      - Dylan16807 4y ago
        The point is, you don't even need to extend the protocol for SRT. You just need to set up servers that are ready, just like companies are in the middle of doing for other techs. Which I'm sure is still significant work, but is it much harder to do with SRT?
        der_rod 4y ago
        > You just need to set up servers that are ready The "just" is doing a lot of work here. Extending existing RTMP infrastructure is obviously much easier than spinning up an entirely different one. And you'd still have specify a way to use anything but MPEG-TS to send audio/video over SRT, which de facto does not exist (at least not that I can find). I agree that extending RTMP isn't ideal, but other solutions have their own set of limitations, problems, and challenges. Whether that's SRT, RIST, or WebRTC. For better and worse, RTMP and dealing with it at scale is well understood.

ofchnofc 4y ago

I don't normally write this type of comment, but you are an absolute powerhouse and I'm so glad to see you pushing WebRTC, WebRTC+OBS, and playing the social game a bit to drum up support. WebRTC has so much potential, I have big ideas for using it in a number of projects, if I can ever just clone myself a couple of times. Thanks for your work and enthusiasm! I constantly dream of a much more simple, but also powerful OBS replacement, with separated components for sending inputs to a "compositing" server, that can then rebroadcast via WebRTC. Maybe with customizable templates that could be more powerful than OBS's scenes. With the ability to let viewers customize/change their views further... It's a bit much to type out all here, but the WebRTC-powered future is very exciting.

toast0 4y ago

> I would love a world where users upload the multiple quality levels. Then anyone could run a stream service, not just those that can afford transcodes. I believe both AV1 and HEVC/h.265 specify optional Scalable Video Coding, although I don't know if that's a common feature of encoders or if this integration supports it. If the stars align, it's straightforward to choose parts of the stream to throw away to fit into available bandwidth or decode capacity without transcoding. Of course, decoder support is also needed.

rcme 4y ago

Does codec negotiation really make sense? Encoding video is so expensive that you can only realistically support one or two codecs at a given time. Anything else is wasteful. > * P2P. Stream your OBS output directly to another user. It is frustrating to see users struggle with setting up RTMP servers/HLS when they want to share their video with just one or two other people. The other use case I really care about is co-streaming. The nginx RTMP plugin supports converting an RTMP stream into an HLS stream. It's pretty easy to set up. Even if OBS used WebRTC or some other p2p solution, you would still need some type of signaling / connection negotiation setup.

Sean-Der 4y ago
> Does codec negotiation really make sense On startup you could be connecting two clients that have completely different sets of codecs. What if you have a low powered device that only has a H264 hardware decoder? What if you have two desktops that prefer AV1. WebRTC lets each side publish what codecs they support, and their priority > It's pretty easy to set up Yea it is not bad if you are comfortable with those things. I want to empower users who have no idea what Linux, codecs or systemd is. I want to create something like 12 year old me would have used. I could have had a lot of fun with my friends if we could have done P2P. > you would still need some type of signaling / connection negotiation setup. I use QR codes for this. For public signaling is also cheap enough that it could be provided for free. I think the project name was simplewebrtc? Hosting RTMP/HLS for free is prohibitively expensive though.
- favsq 4y ago
  >What if you have a low powered device that only has a H264 hardware decoder? What if you have two desktops that prefer AV1. You recode on the server.
  - Sean-Der 4y ago
    You don't have a server (and in my cases you don't want) * I don't want to upload my video from my security camera to a remote host. It has security and privacy implications * I don't have the bandwidth available * I don't want to pay the bandwidth and compute costs. * I don't want to incur the extra latency

MawKKe 4y ago

I've been wondering about that multi track thing w.r.t video live streams Wouldn't it make sense to have separate channels for distinct content streams (e.g game capture, face camera, overlays..) so that you could independently choose/optimize each channel's compression and thus make the whole stream more efficient? I don't know anything about compression algorithms, but I hypothesize that maybe now the algos would also be able to do a better job since the frames have less abrupt discontinuities (in the viewport perspective). This would also enable stream viewers's to activate/deactivate channels when needed. For example, when the overlay/webcam/etc occludes the view on layer (channel) below.

kevincox 4y ago
> so that you could independently choose/optimize each channel's compression and thus make the whole stream more efficient? In general the compression algorithm will make an intelligent decision on where to spend bits. However you may gain a bit more accuracy here. The better win may be to set priorities (maybe make the scrolling text crisp right away because it doesn't move often anyways) but I am still skeptical that you would see much benefit. > enable stream viewers's to activate/deactivate channels when needed On the other hand this would be very cool. I can also see moving away from fixed aspect ratios to moving the bits around the screen. Maybe the webcam can be above the (non-video) chat rather than over top of the video. Or even just mapping the different streams (that I have selected) into my aspect ratio better (like Google Meet and other video conferencing tools do).

zamadatix 4y ago

Even if WebRTC or something that supports all of these is added I still wouldn't be sad we are extending RTMP. If the next hot new thing requires artificially limiting the old thing then it's not really the next hot new thing.

Sean-Der 4y ago
The lack of collective 'internet user' ownership makes me sad. Many people want to improve RTMP, but an open discussion doesn't exist. If RTMP was in the IETF my comment wouldn't apply anymore.
- zamadatix 4y ago
  While not specifically the IETF isn't this spec enhancement done by folks in e.g. the obs, ffmpeg, xsplit, and vlc projects? I.e. is this an ideological concern of which standards body owns it or a practical one of people not being able to improve the standard.