8 ms·

Mediasoup – WebRTC Video Conferencing

73 points by ofrzeta 7y ago

AnonC 7y ago
This website could be a lot more useful if it answered these questions: * How good is this compared to other alternatives? (Or why choose this instead of something else?) * Where are the client apps for this, more importantly on mobile?
- ofrzeta 7y ago
  Sure, the website could be better. Here's a demo https://v3demo.mediasoup.org/ https://v3demo.mediasoup.org/ And here's the source code for the demo: https://github.com/versatica/mediasoup-demo/ https://github.com/versatica/mediasoup-demo/ Libs for native apps: https://github.com/haiyangwu/mediasoup-client-android https://github.com/haiyangwu/mediasoup-client-android https://github.com/ethand91/mediasoup-ios-client https://github.com/ethand91/mediasoup-ios-client
  - anovikov 7y ago
    Libs for native apps suck. But, it is compatible with React Native and works fine there. In general, i recommend it. Only media server that i saw working (slightly) better is Jitsi, but it is 10x more cumbersome and time-consuming to learn.
    - deepinder10 7y ago
      making a single customisation in jitsi is a cumbersome task. You have to accept it how it is.
  - jmillan 7y ago
    I'm a core mediasoup developer. Those libs for native apps are not official mediasoup components. We deliver libmediasoupclient[1] which is a native C++ implementation of the JS mediadiasoup-client[2] and uses libwebrtc[3] C++ native code directly. Those two libs for native apps make use of libmediasoupclient, but they are not written nor maintained by us. Having said that, to the moment there is no official Android or IOS native client but there is libmediasoupclient which will be the core for both. [1]: https://github.com/versatica/libmediasoupclient https://github.com/versatica/libmediasoupclient [2]: https://github.com/versatica/mediasoup-client https://github.com/versatica/mediasoup-client [3]: http://webrtc.github.io/webrtc-org/native-code/ http://webrtc.github.io/webrtc-org/native-code/
- kabes 7y ago
  This is more of a low level API. Not some ready to use thing like Jitsi. That is already a big difference with alternatives. Performance seems good: https://webrtchacks.com/sfu-load-testing/ https://webrtchacks.com/sfu-load-testing/ And it is very actively developed, which is arguably the most important metric in the current webRTC environment. Since the spec and browser implementations are a fast moving target right now and for the years to come.
- ibc 7y ago
  Hi, mediasoup co-author here. Comparing Jitsi with mediasoup is like comparing Netflix (backend + apps) with Express.js + libcurl. Jitsi developers may replace their RTC core internals (including the SFU) with mediasoup + mediasoup-client and you wouldn't even realize of it. Hope this helps.
ac130kz 7y ago
"Cutting-edge" implies that these guys managed to apply secure E2E encryption, but I can't find any references to it on the website. Correct me if I'm wrong.
- supermatt 7y ago
  You are wrong. This is the current state of webrtc. Without considerable hacks (patent-encumbered unaccelerated wasm ffmpeg encoding of pixel data, rolling your own SRTP-like encrypted stream over datachannels and full mesh distribution of keys) this is not currently possible for anything other than a full mesh. Whenever an SFU (effectively a MITM) is involved (needed for more than a small number of participants) e2e encryption is lost. If/when Insertable Streams are commonplace, this will be possible without so many hacks. Edit: I see I was downvoted for correcting you, but there was an article on this very subject yesterday: https://webrtchacks.com/you-dont-have-end-to-end-encryption-e2ee/ https://webrtchacks.com/you-dont-have-end-to-end-encryption-...
  - fulafel 7y ago
    The video transmitted over webrtc srtp is natively dtls encrypted between peers.
    - supermatt 7y ago
      Yes, that is the "full mesh" that I mention in my comment... This platform - and jitsi/janus/zoom/whatever - all use an SFU for more than a handful of participants.
      - fulafel 7y ago
        But thats very doable. And no need for the unaccelerated ffmpeg wasm afaics, and no need to use a "sfu". I don't know what Mediasoup does, was just commenting on what webrtc can do re the "the current state of webrtc" subject.
        7y ago
sandGorgon 7y ago
any comparisons with jitsi ? really would like to understand how these technologies stack against each other. jitsi has been production tested far longer i suppose, through its freely available videoconferencing service https://meet.jit.si/ https://meet.jit.si/
- kabes 7y ago
  Mediasoup has a more low level API. It's more of a library/framework to make your own SFU (although there's an example SFU). Jitsi is more of a ready to use SFU. Mediasoup is newer, but has seen very fast development. Last performance comparison I know of had mediasoup as a clear winner over jitsi. But that's a fast moving target and the usual limitations of a benchmark apply here as well: https://webrtchacks.com/sfu-load-testing/ https://webrtchacks.com/sfu-load-testing/ Mediasoup has a c++ core with node api. Jitsi is java. If you're experienced with webRTC, you will probably like mediasoup better. Otherwise I'd stick with jitsi.
- ibc 7y ago
  mediasoup co-author here. Comparing Jitsi with mediasoup is like comparing Netflix (backend + apps) with Express.js + libcurl. Jitsi developers may replace their RTC core internals (including the SFU) with mediasoup + mediasoup-client and you wouldn't even realize of it. Hope this helps.
  - sandGorgon 7y ago
    Thanks for this. Do you plan to go higher in the stack and release components to get-started-quickly ?
    - ibc 7y ago
      Not at all.
imtringued 7y ago
Can this be used as an alternative to Kurento? I use an ffmpeg command to send an RTP stream to Kurento which then broadcasts that to all clients that connect via WebRTC. I've had to rewrite the code that interacts with Kurento twice already. It doesn't feel like a rock solid solution. e.g. when you accidentally switch audio and video ports in the ffmpeg command the entire kurento instance crashes. I can't expose Kurento to untrusted instances otherwise there is a risk of DOS. I don't feel like running a whole Kurento instance per ffmpeg command is a sustainable solution. It might not be obvious but I am not using WebRTC for video conferencing. My use case is basically regular live streaming but with latencies in the 100ms range. Edit: I just compared the docs https://raw.githubusercontent.com/versatica/mediasoup/v3/art/mediasoup-v3-architecture-01.svg?sanitize=true https://raw.githubusercontent.com/versatica/mediasoup/v3/art... vs https://doc-kurento.readthedocs.io/en/6.13.0/features/kurento_api.html#endpoints https://doc-kurento.readthedocs.io/en/6.13.0/features/kurent... This is exactly what I need!
manthideaal 7y ago
To understand the terms: webrtc, stun, turn, mesh, sfu, mcu, ice and trickle ice, there is (1). 15 minutes to understand what is all this about. What about IPv6 stun and turn?, it seems other people asked the same idea I thought: (2) bout all of this, one is the answer is: As IPv6 takes over the complexity of new networks, STUN and ICE will become irrelevant. I think that with the surge in video conferences and rtc, ipv6 with take off. In my very humble opinion, I would suggest to reserve some address space in IPv6 for rtc, so that a peer is able to adopt a new special ip reserved for rtc. Nothing new under the sun, in 2014 someone comment along this line of thought (2) and (3). So what are we waiting for? (1) https://webrtcglossary.com/ https://webrtcglossary.com/ (2) https://www.quora.com/Will-the-IPv6-result-in-the-death-of-STUN-and-ICE https://www.quora.com/Will-the-IPv6-result-in-the-death-of-S... (3) 2014, AshleysBrain, https://news.ycombinator.com/item?id=7496986 https://news.ycombinator.com/item?id=7496986 I think the solution is IPv6. Once every device on the Internet is uniquely addressable again, we can do away with these NAT hacks and two endpoints should be able to reliably connect to each other again, no matter where they are. Of course, that's assuming we don't get more short-sighted engineering that breaks things again...
- supermatt 7y ago
  addressable !== routeable !== reachable. IPv6 will certainly REDUCE the need for STUN, but there are still (many) cases where you don't want to be "reachable by default", in which case you need a stable reference for negotiating routing and reachability (e.g. STUN).
  - manthideaal 7y ago
    IPv6 could have and address for "reachable by default", I find that very useful, also IPv6 allow many addresses so that is not wasteful.
    - supermatt 7y ago
      Yeah, but if it’s reachable by default then it’s (by definition) open to the world. Otherwise (if you mean routable by default) you will still end up temporarily punching holes in your firewall, which you will need to close afterwards, and possibly recycling your ip so you aren’t still routable on that last used address. sounds like you would personally end up being a STUN server!
pkstn 7y ago
Does it do TURN/STUN?
- ibc 7y ago
  mediasoup is a SFU that must be deployed in a reachable server, so STUN is not needed at all. You may need a TURN server if a client has a restrictive firewall that blocks UDP. mediasoup is not a TURN server but you can deploy a TURN server (i.e. coturn) in your backend.
  - orixilus 7y ago
    I just tried the demo - btw, best UX so far from all that I've tried. It works fine on the same wifi network, but won't connect if one of the devices is using 4G network - is this because the TURN server is not setup? Is it easy to implement that?
    - ibc 7y ago
      The demo is just a demo. mediasoup is a low level library (no UX into mediasoup). The online demo backend does not have any TURN server. It's obviously recommended to deploy a TURN server.
  - supermatt 7y ago
    Are there plans to support a TCP candidate so a TURN server isn't needed at all? It feels a bit wasteful to effectively use a TURN server as a TCP->UDP proxy for a publicly accessible server.
    - ibc 7y ago
      We do support TCP ICE candidates for long time: https://mediasoup.org/documentation/v3/mediasoup/api/#WebRtcTransportOptions https://mediasoup.org/documentation/v3/mediasoup/api/#WebRtc... > TCP candidate so a TURN server isn't needed at all? This is not true. A router may still block TCP traffic different than TLS or traffic that does not have destinatiuon port 80 or 443. So ICE TCP candidates do not avoid the need for a TURN server in certain cases.
ibc 7y ago
Hi, mediasoup co-author here. TL'DR': Pornhub uses mediasoup. I've read many comments here asking about "how mediasoup is different than XXX" or about "mobile apps". I think the Overview in the website should be self explanatory, I'll just paste a fragment here: https://mediasoup.org/documentation/overview/ https://mediasoup.org/documentation/overview/ --------------------- Design goals of mediasoup and its client side libraries: - Be a SFU (Selective Forwarding Unit). - Support both WebRTC and plain RTP input and output. - Be a Node.js module in server side. - Be a tiny JavaScript and C++ libraries in client side. - Be minimalist: just handle the media layer. - Be signaling agnostic: do not mandate any signaling protocol. - Be super low level API. - Support all existing WebRTC endpoints. - Enable integration with well known multimedia libraries/tools. Use cases: - Group video chat applications. - One-to-many (or few-to-many) broadcasting applications in real-time. - RTP streaming.
- bryanrasmussen 7y ago
  I read the whole thing. I am really confused as to why out of that you chose "Pornhub uses mediasoup" as the TLDR?
  - ibc 7y ago
    Just to easily explain that mediasoup is not a replacement for Jitsi or Zoom, but a low level set of libraries for building build different kind of real-time applications, including multi-party videoconference apps (such as Jitsi or Zoom) and others completely different.
deepinder10 7y ago
The performance as mentioned on webrtchacks seems promising, I was planning to go with openvidu but now i am more inclined towards mediasoup. My only concerns are, please provide simple tutorials with demo and code explanation (it would be 30 lines of code) so that beginners like me easily understand it. If you check out openvidu they have provided demos for every use case with explanation. It is hard for a beginner to understand all this stuff and there isn't a js v3 broadcasting demo available. Also if you could mention some performance results on various machines like c5.large, c5.xlarge, c5.2xlarge with number of participants it would be helpful. Thanks a lot

73 points by ofrzeta 7y ago

This website could be a lot more useful if it answered these questions: * How good is this compared to other alternatives? (Or why choose this instead of something else?) * Where are the client apps for this, more importantly on mobile?

ofrzeta 7y ago
Sure, the website could be better. Here's a demo https://v3demo.mediasoup.org/ https://v3demo.mediasoup.org/ And here's the source code for the demo: https://github.com/versatica/mediasoup-demo/ https://github.com/versatica/mediasoup-demo/ Libs for native apps: https://github.com/haiyangwu/mediasoup-client-android https://github.com/haiyangwu/mediasoup-client-android https://github.com/ethand91/mediasoup-ios-client https://github.com/ethand91/mediasoup-ios-client
- anovikov 7y ago
  Libs for native apps suck. But, it is compatible with React Native and works fine there. In general, i recommend it. Only media server that i saw working (slightly) better is Jitsi, but it is 10x more cumbersome and time-consuming to learn.
  - deepinder10 7y ago
    making a single customisation in jitsi is a cumbersome task. You have to accept it how it is.
- jmillan 7y ago
  I'm a core mediasoup developer. Those libs for native apps are not official mediasoup components. We deliver libmediasoupclient[1] which is a native C++ implementation of the JS mediadiasoup-client[2] and uses libwebrtc[3] C++ native code directly. Those two libs for native apps make use of libmediasoupclient, but they are not written nor maintained by us. Having said that, to the moment there is no official Android or IOS native client but there is libmediasoupclient which will be the core for both. [1]: https://github.com/versatica/libmediasoupclient https://github.com/versatica/libmediasoupclient [2]: https://github.com/versatica/mediasoup-client https://github.com/versatica/mediasoup-client [3]: http://webrtc.github.io/webrtc-org/native-code/ http://webrtc.github.io/webrtc-org/native-code/
kabes 7y ago
This is more of a low level API. Not some ready to use thing like Jitsi. That is already a big difference with alternatives. Performance seems good: https://webrtchacks.com/sfu-load-testing/ https://webrtchacks.com/sfu-load-testing/ And it is very actively developed, which is arguably the most important metric in the current webRTC environment. Since the spec and browser implementations are a fast moving target right now and for the years to come.
ibc 7y ago
Hi, mediasoup co-author here. Comparing Jitsi with mediasoup is like comparing Netflix (backend + apps) with Express.js + libcurl. Jitsi developers may replace their RTC core internals (including the SFU) with mediasoup + mediasoup-client and you wouldn't even realize of it. Hope this helps.

ac130kz 7y ago

"Cutting-edge" implies that these guys managed to apply secure E2E encryption, but I can't find any references to it on the website. Correct me if I'm wrong.

supermatt 7y ago
You are wrong. This is the current state of webrtc. Without considerable hacks (patent-encumbered unaccelerated wasm ffmpeg encoding of pixel data, rolling your own SRTP-like encrypted stream over datachannels and full mesh distribution of keys) this is not currently possible for anything other than a full mesh. Whenever an SFU (effectively a MITM) is involved (needed for more than a small number of participants) e2e encryption is lost. If/when Insertable Streams are commonplace, this will be possible without so many hacks. Edit: I see I was downvoted for correcting you, but there was an article on this very subject yesterday: https://webrtchacks.com/you-dont-have-end-to-end-encryption-e2ee/ https://webrtchacks.com/you-dont-have-end-to-end-encryption-...
- fulafel 7y ago
  The video transmitted over webrtc srtp is natively dtls encrypted between peers.
  - supermatt 7y ago
    Yes, that is the "full mesh" that I mention in my comment... This platform - and jitsi/janus/zoom/whatever - all use an SFU for more than a handful of participants.
    - fulafel 7y ago
      But thats very doable. And no need for the unaccelerated ffmpeg wasm afaics, and no need to use a "sfu". I don't know what Mediasoup does, was just commenting on what webrtc can do re the "the current state of webrtc" subject.
      - 7y ago

sandGorgon 7y ago

any comparisons with jitsi ? really would like to understand how these technologies stack against each other. jitsi has been production tested far longer i suppose, through its freely available videoconferencing service https://meet.jit.si/ https://meet.jit.si/

kabes 7y ago
Mediasoup has a more low level API. It's more of a library/framework to make your own SFU (although there's an example SFU). Jitsi is more of a ready to use SFU. Mediasoup is newer, but has seen very fast development. Last performance comparison I know of had mediasoup as a clear winner over jitsi. But that's a fast moving target and the usual limitations of a benchmark apply here as well: https://webrtchacks.com/sfu-load-testing/ https://webrtchacks.com/sfu-load-testing/ Mediasoup has a c++ core with node api. Jitsi is java. If you're experienced with webRTC, you will probably like mediasoup better. Otherwise I'd stick with jitsi.
ibc 7y ago
mediasoup co-author here. Comparing Jitsi with mediasoup is like comparing Netflix (backend + apps) with Express.js + libcurl. Jitsi developers may replace their RTC core internals (including the SFU) with mediasoup + mediasoup-client and you wouldn't even realize of it. Hope this helps.
- sandGorgon 7y ago
  Thanks for this. Do you plan to go higher in the stack and release components to get-started-quickly ?
  - ibc 7y ago
    Not at all.

imtringued 7y ago

Can this be used as an alternative to Kurento? I use an ffmpeg command to send an RTP stream to Kurento which then broadcasts that to all clients that connect via WebRTC. I've had to rewrite the code that interacts with Kurento twice already. It doesn't feel like a rock solid solution. e.g. when you accidentally switch audio and video ports in the ffmpeg command the entire kurento instance crashes. I can't expose Kurento to untrusted instances otherwise there is a risk of DOS. I don't feel like running a whole Kurento instance per ffmpeg command is a sustainable solution. It might not be obvious but I am not using WebRTC for video conferencing. My use case is basically regular live streaming but with latencies in the 100ms range. Edit: I just compared the docs https://raw.githubusercontent.com/versatica/mediasoup/v3/art/mediasoup-v3-architecture-01.svg?sanitize=true https://raw.githubusercontent.com/versatica/mediasoup/v3/art... vs https://doc-kurento.readthedocs.io/en/6.13.0/features/kurento_api.html#endpoints https://doc-kurento.readthedocs.io/en/6.13.0/features/kurent... This is exactly what I need!

manthideaal 7y ago

To understand the terms: webrtc, stun, turn, mesh, sfu, mcu, ice and trickle ice, there is (1). 15 minutes to understand what is all this about. What about IPv6 stun and turn?, it seems other people asked the same idea I thought: (2) bout all of this, one is the answer is: As IPv6 takes over the complexity of new networks, STUN and ICE will become irrelevant. I think that with the surge in video conferences and rtc, ipv6 with take off. In my very humble opinion, I would suggest to reserve some address space in IPv6 for rtc, so that a peer is able to adopt a new special ip reserved for rtc. Nothing new under the sun, in 2014 someone comment along this line of thought (2) and (3). So what are we waiting for? (1) https://webrtcglossary.com/ https://webrtcglossary.com/ (2) https://www.quora.com/Will-the-IPv6-result-in-the-death-of-STUN-and-ICE https://www.quora.com/Will-the-IPv6-result-in-the-death-of-S... (3) 2014, AshleysBrain, https://news.ycombinator.com/item?id=7496986 https://news.ycombinator.com/item?id=7496986 I think the solution is IPv6. Once every device on the Internet is uniquely addressable again, we can do away with these NAT hacks and two endpoints should be able to reliably connect to each other again, no matter where they are. Of course, that's assuming we don't get more short-sighted engineering that breaks things again...

supermatt 7y ago
addressable !== routeable !== reachable. IPv6 will certainly REDUCE the need for STUN, but there are still (many) cases where you don't want to be "reachable by default", in which case you need a stable reference for negotiating routing and reachability (e.g. STUN).
- manthideaal 7y ago
  IPv6 could have and address for "reachable by default", I find that very useful, also IPv6 allow many addresses so that is not wasteful.
  - supermatt 7y ago
    Yeah, but if it’s reachable by default then it’s (by definition) open to the world. Otherwise (if you mean routable by default) you will still end up temporarily punching holes in your firewall, which you will need to close afterwards, and possibly recycling your ip so you aren’t still routable on that last used address. sounds like you would personally end up being a STUN server!

pkstn 7y ago

Does it do TURN/STUN?

ibc 7y ago
mediasoup is a SFU that must be deployed in a reachable server, so STUN is not needed at all. You may need a TURN server if a client has a restrictive firewall that blocks UDP. mediasoup is not a TURN server but you can deploy a TURN server (i.e. coturn) in your backend.
- orixilus 7y ago
  I just tried the demo - btw, best UX so far from all that I've tried. It works fine on the same wifi network, but won't connect if one of the devices is using 4G network - is this because the TURN server is not setup? Is it easy to implement that?
  - ibc 7y ago
    The demo is just a demo. mediasoup is a low level library (no UX into mediasoup). The online demo backend does not have any TURN server. It's obviously recommended to deploy a TURN server.
- supermatt 7y ago
  Are there plans to support a TCP candidate so a TURN server isn't needed at all? It feels a bit wasteful to effectively use a TURN server as a TCP->UDP proxy for a publicly accessible server.
  - ibc 7y ago
    We do support TCP ICE candidates for long time: https://mediasoup.org/documentation/v3/mediasoup/api/#WebRtcTransportOptions https://mediasoup.org/documentation/v3/mediasoup/api/#WebRtc... > TCP candidate so a TURN server isn't needed at all? This is not true. A router may still block TCP traffic different than TLS or traffic that does not have destinatiuon port 80 or 443. So ICE TCP candidates do not avoid the need for a TURN server in certain cases.

ibc 7y ago

Hi, mediasoup co-author here. TL'DR': Pornhub uses mediasoup. I've read many comments here asking about "how mediasoup is different than XXX" or about "mobile apps". I think the Overview in the website should be self explanatory, I'll just paste a fragment here: https://mediasoup.org/documentation/overview/ https://mediasoup.org/documentation/overview/ --------------------- Design goals of mediasoup and its client side libraries: - Be a SFU (Selective Forwarding Unit). - Support both WebRTC and plain RTP input and output. - Be a Node.js module in server side. - Be a tiny JavaScript and C++ libraries in client side. - Be minimalist: just handle the media layer. - Be signaling agnostic: do not mandate any signaling protocol. - Be super low level API. - Support all existing WebRTC endpoints. - Enable integration with well known multimedia libraries/tools. Use cases: - Group video chat applications. - One-to-many (or few-to-many) broadcasting applications in real-time. - RTP streaming.

bryanrasmussen 7y ago
I read the whole thing. I am really confused as to why out of that you chose "Pornhub uses mediasoup" as the TLDR?
- ibc 7y ago
  Just to easily explain that mediasoup is not a replacement for Jitsi or Zoom, but a low level set of libraries for building build different kind of real-time applications, including multi-party videoconference apps (such as Jitsi or Zoom) and others completely different.

deepinder10 7y ago

The performance as mentioned on webrtchacks seems promising, I was planning to go with openvidu but now i am more inclined towards mediasoup. My only concerns are, please provide simple tutorials with demo and code explanation (it would be 30 lines of code) so that beginners like me easily understand it. If you check out openvidu they have provided demos for every use case with explanation. It is hard for a beginner to understand all this stuff and there isn't a js v3 broadcasting demo available. Also if you could mention some performance results on various machines like c5.large, c5.xlarge, c5.2xlarge with number of participants it would be helpful. Thanks a lot