3 ms·

- You need 3 clients, minimum—Web, iOS, Android—or you're not an alternative. None of them can be way different from or far worse than the others. - Like 90% o

- You need 3 clients, minimum—Web, iOS, Android—or you're not an alternative. None of them can be way different from or far worse than the others. - Like 90% of the benefit of Zoom (and other large video & voice chat providers) is their backend infrastructure. You cannot justify replicating that for a small single-customer or small group installation. It'd be far too expensive. So your open-source solution is ~always going to be worse. ("We've also had mixed results testing with team members far away on rough connections, where audio/video was workable in zoom but sometimes unworkable with our internal setup.") - "they can't really do department meetings or let alone an all-hands type of meeting with ~1000 people on the call." Those should be broadcast as one-way streaming video, anyway. Set it up like a live video "podcast" or twitch stream with text chat for questions from the audience, if needed.

djohnston 4y ago
I'm not familiar with the nitty gritty of video streaming infra but I'm curious, with the crazy array of services on AWS/most other cloud providers, what in particular is missing or cannot scale down cost efficiently?
- everforward 4y ago
  I would guess a big part is switching/routing specifically geared towards low-latency audio/video. Just as an example, they probably use UDP for latency, which has no ordering as part of the protocol. However, for streaming stuff, you really want to drop out of order packets and prioritize whatever the latest packet is. Zoom probably has some kind of custom routing that handles that. If they get congested or clients are unstable, it prioritizes serving the latest audio/video rather than trying to push things out in the order they came in, only for the Zoom client to discard the out of order packets anyways. Amazon can't really do that, because they don't know which bits of data are the ordering. On the pricing side, people are probably the component that scales the least. You're probably going to need a dedicated person to monitor/fix/upgrade it, and deal with calls from people who have issues. Between salary and benefits, that's at least a 6 figure expense. Zoom's most expensive tier is $240/license/year, so $100,000 would buy you 416 licenses. And $100,000 is a lowball here; that's roundabouts a $60k salary, and the other $40k is paid out in benefits/space/etc. This also doesn't count the bandwidth charges, nor the hardware to run the system on. Imo, it's also just generally a bad idea to DIY anything business-critical that isn't a core competency of the company. It tends to cause issues. What do you do when the person that built this bespoke Zoom alternative quits? Can you risk it breaking and not working until you can hire someone new and they figure out how it works? You could hire 2 of them, but Zoom is definitely cheaper then. What do you do if the open source project gets abandoned? It also distracts from things that actually make you money. It makes more sense to me to focus on your core line of business, and use the additional proceeds to just pay for Zoom.
- jech 4y ago
  There's nothing particularly difficult on the server side — a quality SFU should be capable to handle on the order of 400 video flows per core, and there are quite a few high-quality free software SFUs available (Janus, Jitsi, ion-sfu, livekit, Galene). To give some perspective: we're using Galene for lectures, and our single-CPU server uses around 40% CPU usage in a room with 120 students (who keep their cameras switched off during the lecture, of course, and only occasionally switch them on to ask questions). As the grandparent mentioned, the problem is the client side. Since there is no standard videoconferencing protocol, every free software project needs to develop their own clients. And it's difficult for a free software project to have the manpower and expertise to develop quality clients for the web, Android and iOS, so in effect what we currently have are mostly half-baked web clients. There is some hope, though. The IETF have been working on standard protocols for ingress (https://datatracker.ietf.org/wg/wish/ https://datatracker.ietf.org/wg/wish/), and if their protocols get deployed, you'll be able to use the same streaming software (think OBS) or IP camera with multiple distinct videoconferencing servers. An interoperable interactive videoconferencing protocol is nowhere near, but as more people understand videoconferencing technology, there is some hope that people will get together and start working on multi-protocol clients (remember Pidgin?). Full disclosure: I'm the author of Galene (https://galene.org https://galene.org), and I've been actively participating in the Pion community (https://github.com/pion/webrtc https://github.com/pion/webrtc) and collaborating with the authors of ion-sfu (https://github.com/pion/ion-sfu https://github.com/pion/ion-sfu) and LiveKit (https://github.com/livekit https://github.com/livekit).
montroser 4y ago
We've talked about broadcast calls with text chat for Q&A, and it could probably be made to work... But, they just want it to all be the same platform, not require tech-averse execs to learn new tools, not have to work out the logistics of who needs to be on the "real" call each time, emoji reactions in the moment, etc. They do these town-hall sessions where they want to be able to have back forth with employees asking questions. At the end of the day, it's a tough proposition to say, "you can either have the features you've come to rely on, or you can have your privacy and control". But I guess that is where we are.

djohnston 4y ago

I'm not familiar with the nitty gritty of video streaming infra but I'm curious, with the crazy array of services on AWS/most other cloud providers, what in particular is missing or cannot scale down cost efficiently?

everforward 4y ago
I would guess a big part is switching/routing specifically geared towards low-latency audio/video. Just as an example, they probably use UDP for latency, which has no ordering as part of the protocol. However, for streaming stuff, you really want to drop out of order packets and prioritize whatever the latest packet is. Zoom probably has some kind of custom routing that handles that. If they get congested or clients are unstable, it prioritizes serving the latest audio/video rather than trying to push things out in the order they came in, only for the Zoom client to discard the out of order packets anyways. Amazon can't really do that, because they don't know which bits of data are the ordering. On the pricing side, people are probably the component that scales the least. You're probably going to need a dedicated person to monitor/fix/upgrade it, and deal with calls from people who have issues. Between salary and benefits, that's at least a 6 figure expense. Zoom's most expensive tier is $240/license/year, so $100,000 would buy you 416 licenses. And $100,000 is a lowball here; that's roundabouts a $60k salary, and the other $40k is paid out in benefits/space/etc. This also doesn't count the bandwidth charges, nor the hardware to run the system on. Imo, it's also just generally a bad idea to DIY anything business-critical that isn't a core competency of the company. It tends to cause issues. What do you do when the person that built this bespoke Zoom alternative quits? Can you risk it breaking and not working until you can hire someone new and they figure out how it works? You could hire 2 of them, but Zoom is definitely cheaper then. What do you do if the open source project gets abandoned? It also distracts from things that actually make you money. It makes more sense to me to focus on your core line of business, and use the additional proceeds to just pay for Zoom.
jech 4y ago
There's nothing particularly difficult on the server side — a quality SFU should be capable to handle on the order of 400 video flows per core, and there are quite a few high-quality free software SFUs available (Janus, Jitsi, ion-sfu, livekit, Galene). To give some perspective: we're using Galene for lectures, and our single-CPU server uses around 40% CPU usage in a room with 120 students (who keep their cameras switched off during the lecture, of course, and only occasionally switch them on to ask questions). As the grandparent mentioned, the problem is the client side. Since there is no standard videoconferencing protocol, every free software project needs to develop their own clients. And it's difficult for a free software project to have the manpower and expertise to develop quality clients for the web, Android and iOS, so in effect what we currently have are mostly half-baked web clients. There is some hope, though. The IETF have been working on standard protocols for ingress (https://datatracker.ietf.org/wg/wish/ https://datatracker.ietf.org/wg/wish/), and if their protocols get deployed, you'll be able to use the same streaming software (think OBS) or IP camera with multiple distinct videoconferencing servers. An interoperable interactive videoconferencing protocol is nowhere near, but as more people understand videoconferencing technology, there is some hope that people will get together and start working on multi-protocol clients (remember Pidgin?). Full disclosure: I'm the author of Galene (https://galene.org https://galene.org), and I've been actively participating in the Pion community (https://github.com/pion/webrtc https://github.com/pion/webrtc) and collaborating with the authors of ion-sfu (https://github.com/pion/ion-sfu https://github.com/pion/ion-sfu) and LiveKit (https://github.com/livekit https://github.com/livekit).

montroser 4y ago

We've talked about broadcast calls with text chat for Q&A, and it could probably be made to work... But, they just want it to all be the same platform, not require tech-averse execs to learn new tools, not have to work out the logistics of who needs to be on the "real" call each time, emoji reactions in the moment, etc. They do these town-hall sessions where they want to be able to have back forth with employees asking questions. At the end of the day, it's a tough proposition to say, "you can either have the features you've come to rely on, or you can have your privacy and control". But I guess that is where we are.