3 ms·

> I wonder if this setup can be leveraged to create a Twitch-like streaming solution based on Matrix. Yes, we've built it with this in mind. There are three p

> I wonder if this setup can be leveraged to create a Twitch-like streaming solution based on Matrix. Yes, we've built it with this in mind. There are three possible approaches: * You can broadcast a headless client via HLS or RTMP, similar to how Jibri works for Jitsi, and how we broadcast FOSDEM (https://matrix.org/blog/2021/02/15/how-we-hosted-fosdem-2021-on-matrix/ https://matrix.org/blog/2021/02/15/how-we-hosted-fosdem-2021...). Basically you run a headless Chrome against a virtual X server and pipe X's framebuffer into ffmpeg. It's a pretty blunt approach and obviously uses a lot of serverside resources, but ensures that the broadcasted stream is completely faithful given that it's literally a recording of what a client would be seeing. * Alternatively, you could do the same approach, but pipe it into a WebRTC broadcasting platform rather than HLS/RTMP/RTSP/DASH. This could give much lower latency, but more bandwidth on the server given you're effectively setting up a 1:1 VoIP call with each viewer, and you'll be unlikely to be able to use CDNs (unless we end up in some crazy world where freeform multicast on the internet finally comes to pass thanks to CDNs fanning out your RTP packets for you :P) * Finally, you could composite the streams together serverside using an MSC3401 compliant MCU and broadcast the result via one or other approach. This might not be quite as high fidelity as running a 'real' Element Call instance serverside, but could be way more efficient in terms of resources, and could prove perfectly adequate. The only catch is that we haven't bolted MSC3401 onto any MCUs yet. (Clearly we should get back in touch with our friends at FreeSWITCH / Signalwire :D) > What I'm also curious about is how this fits in with the experiments on the peer-to-peer mesh system (running a homeserver on every client). Forwarding video across a web of clients would become bandwidth and CPU intensive real fast and the added latency would be non-negligible. MSC3401 is pretty much orthogonal to P2P Matrix. The way it works is that it'll use whatever conferencing nodes ('foci') are advertised in the room to mix together the calls in a decentralised fashion. If there are none (as per today) then it just goes full mesh. If there is one, then everyone will converge on it. If there are two, then folks will pick the one with the lowest latency. But critically, it doesn't matter whether the clients are talking to a serverside homeserver or are running P2P Matrix. Finally, the foci could run either serverside or clientside (much as skype supernodes were effectively clientside foci), but we need to get serverside foci working first before we add clientside ones in for the P2P crew ;)

anticensor 5y ago
> If there are two, then folks will pick the one with the lowest latency. No it needs to be deterministic. "Just picking the lowest latency node" would result in sync failures i.e. not everybody hearing the same thing.
- Arathorn 5y ago
  Mixed sync +/- 100ms of latency or whatever is probably fine, for most purposes? If you really want everyone to be centralised on a single SFU then you’d set the permissions on the room appropriately.
  - anticensor 5y ago
    It can be distributed SFU just the relative timings need to converge deterministically (possibly by servers agreeing on a virtual apex to artificially delay the streams to make up for topological differences) to make everyone hear the same thing.
    - Arathorn 5y ago
      So if it turns out that the streams need to be synced then we can delay them to do so - but i am a bit unsure on how important that really is. Are you worried about folks talking over each other if they think they successfully interrupted but then a remote user thought the same and they collide and backoff, like csma/cd?
      - anticensor 5y ago
        > Are you worried about folks talking over each other if they think they successfully interrupted but then a remote user thought the same and they collide and backoff, like csma/cd? Yes, that is really important for classroom use cases.

anticensor 5y ago

> If there are two, then folks will pick the one with the lowest latency. No it needs to be deterministic. "Just picking the lowest latency node" would result in sync failures i.e. not everybody hearing the same thing.

Arathorn 5y ago
Mixed sync +/- 100ms of latency or whatever is probably fine, for most purposes? If you really want everyone to be centralised on a single SFU then you’d set the permissions on the room appropriately.
- anticensor 5y ago
  It can be distributed SFU just the relative timings need to converge deterministically (possibly by servers agreeing on a virtual apex to artificially delay the streams to make up for topological differences) to make everyone hear the same thing.
  - Arathorn 5y ago
    So if it turns out that the streams need to be synced then we can delay them to do so - but i am a bit unsure on how important that really is. Are you worried about folks talking over each other if they think they successfully interrupted but then a remote user thought the same and they collide and backoff, like csma/cd?
    - anticensor 5y ago
      > Are you worried about folks talking over each other if they think they successfully interrupted but then a remote user thought the same and they collide and backoff, like csma/cd? Yes, that is really important for classroom use cases.