5 ms·

I want to like Jitsi, but I can't. I've been testing it out on a variety of servers, bare-metal and cloud with Debian and CentOS. Regardless of platform, it doe

I want to like Jitsi, but I can't. I've been testing it out on a variety of servers, bare-metal and cloud with Debian and CentOS. Regardless of platform, it doesn't scale, it eats memory like peanuts, and can saturate even 10GbE network connections. The service, as opposed to the server, clearly works well. But, the service doesn't have anything like the load that Zoom, Teams, Hangouts, etc. must deal with.

_-___________-_ 7y ago
I've been helping quite a few people set up Jitsi lately. The software works reasonably well, although documentation is lacking and a lot of configuration options are named terribly, but one thing I've noticed is people's expectations about bandwidth usage are way lower than actuality, especially outbound from the server. But a bit of napkin math suggests that Jitsi isn't doing anything fundamentally inefficient here; one high-res stream plus N low-res streams transmitted to N participants is just a lot of bandwidth.
- nemoniac 7y ago
  Please tell us what the config options should be.
  - _-___________-_ 7y ago
    Config options for what?
    - BrowserMeeting 7y ago
      lol. Seriously, for what?
- olgs 7y ago
  This is so very true. We have been monitoring the bandwidth, CPU and mempools of our small Jitsi Meet instance living on an ESXi VM on a Mac Mini. We notice that 3-4 participants typically consume 5-7mbps downstream to the server and about 5mbps upstream from the server while CPU usage stay in the 15-20% range. The bandwidth scales fairly quickly with the number of participants, especially on phone type devices.
jka 7y ago
It could be worth reporting some feedback to them directly, if you haven't already? They're quite responsive on GitHub ( https://github.com/jitsi/jitsi-meet https://github.com/jitsi/jitsi-meet ) and I'm sure additional details regarding any bottlenecks would be appreciated. Identifying even modest improvements now could result in large memory and bandwidth resource savings over the next few weeks and months.
- CrankyBear 7y ago
  Done. I'm working with them to hammer things out.
  - reitzensteinm 7y ago
    Do you have any Firefox users? Not having simulcast basically makes it impossible to scale up to large meetings.
    - CrankyBear 7y ago
      Good to know.
manquer 7y ago
What is you setup look like ? If you have largely 1:1 meetings you can enable p2p mode only , video will directly be shared between users. I have had no problems delivering 100-150 concurrent sessions at ~1 Gbps per bridge, beyond that we usually cluster the bridge ( Octo is out of the box solution jitsi uses for this)
xiii1408 7y ago
Why is that? Is each of the participants audio/video going over the server so that you need p^2 bandwidth? Are there cool P2P tricks they are/could be using? I've been wondering about videoconferencing scale, since Zoom seems to have handled a huge explosion in usage very well. Are they just very good at autoscaling AWS instances, or do they use cool tricks to reduce bandwidth?
- quicklyfrozen 7y ago
  The server is receiving 1 stream per participant, and forwarding that stream to all other participants. So for say 10 participants, that's 10 inbound and 90 outbound. If you switched to p2p instead, then each participant would need to send 10 streams instead of 1, and most people's upload bandwidth is much lower than their download (at least in the US). It's possible for the server to make decisions on which video streams to forward (e.g. last x talkers) to reduce the number of outbound streams, but switching streams takes a bit of time (you'll need to get a new iframe from the just-switched-on participant) which affects the interactivity of the session. Modern video codecs also allow for layering of into progressive levels of enhancement, so lower detail versions of the stream can be forwarded to participants will lower bandwidth without the server needing to transcode anything. (Not sure if Jitsi has implemented anything like that.)
  - jakear 7y ago
    Couldn't the server compose the streams itself? Perhaps as simple as tiling them all together and the client can manage that stream, or request single high quality streams if needed. There'd be a CPU/bandwidth tradeoff being made here, I wonder how the prices on standard cloud providers would compare.
    - quicklyfrozen 7y ago
      Sure, but that adds a lot of CPU load server side (and cloud CPU is still fairly expensive), adds latency for the extra buffering, decoding, and re-encoding, and limits the display options available client-side.
fleetside72 7y ago
I haven't tried this at scale, but was pleased with 6 participants on my $200 r710 hardware with 10megabits upload connection service. Memory usage is minimal, couple GB tops. Not running high-res, but was good enough. I mean they develop this and make it freely available for self-host, I'm pleased.
spditner 7y ago
There are a few of things to be aware of when deploying your own instance in jitsi-meet/config.js: Firefox simulcast is still experimental (edit: and disabled by default), so Firefox is only sending HD to the video-bridge and no LD stream. The HD is then relayed to everyone even as a thumbnail. If you don't need to see everyone's video all the time, set channelLastN: 5 or a similar number, and only the last N speakers video will be broadcast If you don't need 720p (1280x720), change the constraints: video section to be something like 360p (640x360) Enable layer suspension so that HD is not sent to the server when not needed: enableLayerSuspension: true
rcarmo 7y ago
I've been running multiple deployments of it on Azure, on VMs with as low as 1 core/2 GB. I literally set it up every few days from scratch using https://github.com/rcarmo/azure-ubuntu-jitsi https://github.com/rcarmo/azure-ubuntu-jitsi For my use case (around 10-15 friends, for informal meet-ups) it has worked spectacularly well.
- bronco21016 7y ago
  I hesitated to go through the hassle of setting this up because I was worried such a low-end VM wouldn’t be able to handle ~10 users. Sounds like any of the $5/month VMs from the likes of DO, Linode, etc should handle this. Luckily there is a small hoster with a data center near me with similar specs/price. Their free offering on AWS works fine but it would be nice to get lower latency and have my own domain. Thanks for sharing!
BrowserMeeting 7y ago
Video stream, especially if it a group setting is always expensive. I can’t understand why people have such high expectations for video conferencing. This is the reason why Zoom and Microsoft Teams is killing it: video conferencing is inherently difficult and not a trivial task.

_-___________-_ 7y ago

I've been helping quite a few people set up Jitsi lately. The software works reasonably well, although documentation is lacking and a lot of configuration options are named terribly, but one thing I've noticed is people's expectations about bandwidth usage are way lower than actuality, especially outbound from the server. But a bit of napkin math suggests that Jitsi isn't doing anything fundamentally inefficient here; one high-res stream plus N low-res streams transmitted to N participants is just a lot of bandwidth.

nemoniac 7y ago
Please tell us what the config options should be.
- _-___________-_ 7y ago
  Config options for what?
  - BrowserMeeting 7y ago
    lol. Seriously, for what?
olgs 7y ago
This is so very true. We have been monitoring the bandwidth, CPU and mempools of our small Jitsi Meet instance living on an ESXi VM on a Mac Mini. We notice that 3-4 participants typically consume 5-7mbps downstream to the server and about 5mbps upstream from the server while CPU usage stay in the 15-20% range. The bandwidth scales fairly quickly with the number of participants, especially on phone type devices.

jka 7y ago

It could be worth reporting some feedback to them directly, if you haven't already? They're quite responsive on GitHub ( https://github.com/jitsi/jitsi-meet https://github.com/jitsi/jitsi-meet ) and I'm sure additional details regarding any bottlenecks would be appreciated. Identifying even modest improvements now could result in large memory and bandwidth resource savings over the next few weeks and months.

CrankyBear 7y ago
Done. I'm working with them to hammer things out.
- reitzensteinm 7y ago
  Do you have any Firefox users? Not having simulcast basically makes it impossible to scale up to large meetings.
  - CrankyBear 7y ago
    Good to know.

manquer 7y ago

What is you setup look like ? If you have largely 1:1 meetings you can enable p2p mode only , video will directly be shared between users. I have had no problems delivering 100-150 concurrent sessions at ~1 Gbps per bridge, beyond that we usually cluster the bridge ( Octo is out of the box solution jitsi uses for this)

xiii1408 7y ago

Why is that? Is each of the participants audio/video going over the server so that you need p^2 bandwidth? Are there cool P2P tricks they are/could be using? I've been wondering about videoconferencing scale, since Zoom seems to have handled a huge explosion in usage very well. Are they just very good at autoscaling AWS instances, or do they use cool tricks to reduce bandwidth?

quicklyfrozen 7y ago
The server is receiving 1 stream per participant, and forwarding that stream to all other participants. So for say 10 participants, that's 10 inbound and 90 outbound. If you switched to p2p instead, then each participant would need to send 10 streams instead of 1, and most people's upload bandwidth is much lower than their download (at least in the US). It's possible for the server to make decisions on which video streams to forward (e.g. last x talkers) to reduce the number of outbound streams, but switching streams takes a bit of time (you'll need to get a new iframe from the just-switched-on participant) which affects the interactivity of the session. Modern video codecs also allow for layering of into progressive levels of enhancement, so lower detail versions of the stream can be forwarded to participants will lower bandwidth without the server needing to transcode anything. (Not sure if Jitsi has implemented anything like that.)
- jakear 7y ago
  Couldn't the server compose the streams itself? Perhaps as simple as tiling them all together and the client can manage that stream, or request single high quality streams if needed. There'd be a CPU/bandwidth tradeoff being made here, I wonder how the prices on standard cloud providers would compare.
  - quicklyfrozen 7y ago
    Sure, but that adds a lot of CPU load server side (and cloud CPU is still fairly expensive), adds latency for the extra buffering, decoding, and re-encoding, and limits the display options available client-side.

fleetside72 7y ago

I haven't tried this at scale, but was pleased with 6 participants on my $200 r710 hardware with 10megabits upload connection service. Memory usage is minimal, couple GB tops. Not running high-res, but was good enough. I mean they develop this and make it freely available for self-host, I'm pleased.

spditner 7y ago

There are a few of things to be aware of when deploying your own instance in jitsi-meet/config.js: Firefox simulcast is still experimental (edit: and disabled by default), so Firefox is only sending HD to the video-bridge and no LD stream. The HD is then relayed to everyone even as a thumbnail. If you don't need to see everyone's video all the time, set channelLastN: 5 or a similar number, and only the last N speakers video will be broadcast If you don't need 720p (1280x720), change the constraints: video section to be something like 360p (640x360) Enable layer suspension so that HD is not sent to the server when not needed: enableLayerSuspension: true

rcarmo 7y ago

I've been running multiple deployments of it on Azure, on VMs with as low as 1 core/2 GB. I literally set it up every few days from scratch using https://github.com/rcarmo/azure-ubuntu-jitsi https://github.com/rcarmo/azure-ubuntu-jitsi For my use case (around 10-15 friends, for informal meet-ups) it has worked spectacularly well.

bronco21016 7y ago
I hesitated to go through the hassle of setting this up because I was worried such a low-end VM wouldn’t be able to handle ~10 users. Sounds like any of the $5/month VMs from the likes of DO, Linode, etc should handle this. Luckily there is a small hoster with a data center near me with similar specs/price. Their free offering on AWS works fine but it would be nice to get lower latency and have my own domain. Thanks for sharing!

BrowserMeeting 7y ago

Video stream, especially if it a group setting is always expensive. I can’t understand why people have such high expectations for video conferencing. This is the reason why Zoom and Microsoft Teams is killing it: video conferencing is inherently difficult and not a trivial task.