9 ms·
Enhanced noise suppression in Jitsi Meet
- dsr_ 4y agoAssuming the demo samples aren't rigged, that's a very substantial improvement.
- tbalsam 4y agoVery very good, a little bit of stuttering during the honking I think but I like it overall! :D :) Jitsi Meet has been a great alternative to other meeting apps in these crazy times.
- Kwpolska 4y agoMy experience with Jitsi Meet has been quite bad. My previous employer was a cheapskate, and they self-hosted Jitsi Meet. Random disconnections and instability were pretty much a daily occurrence, some people were disconnected every few seconds. While I suppose the self-hosting by Cheapskate Inc. was the main culprit, Jitsi’s screen sharing wasn’t looking very good.
- spockz 4y agoAside from consuming a ton of resources when screen sharing, my experience with Jitsi meet has been very good. It consumes two cores of my 5900X (1 for the Firefox process, and another for some system process I don’t recall exactly) but it works. This was with sharing a 4K screen. I have run jitsi on cheap VMs and it worked decently. But you need quite some cores to serve all the traffic. Ultimately I ended up having as many 2-4core VMs as I had concurrent calls.
- 2Gkashmiri 4y agohow is the meet.jit.si hosted? i assume with lots and lots of random users, the bandwidth and processing costs to be astronomical
- porjo 4y agoFor a large proportion of users I expect the audio/video to be peer to peer thanks to WebRTC. Other traffic (for that set of users) should be minimal.
- troyvit 4y agoMy last employers were cheapskates too (I love 'em for it) and they just used meet.jit.si for calls. It was a lot more stable than self-hosted jitsi. That said there were almost always microphone or video issues using it, just because people weren't used to it I guess. It made job interviews fun. It was a nice live test to show how a potential employee handled adversity.
- saghul 4y agoWe’ve made significant tweaks to screen-sharing in the past 2-3 stable releases, in case you feel inclined to check us out again :-)
- TingPing 4y agoMy company self-hosts an instance and it's excellent.
- andrepd 4y agoSo someone hosted $software on a shitty server, and you blame $software for the shitty performance? To draw any conclusions you should look at meet.jit.si (hosted by Jitsi), no?
- shaan7 4y agoIndeed! I recently used a locally hosted Jitsi to talk to my family in the other room while in COVID isolation. It was a life saver, and extremely easy to setup with docker-compose with only a handful of steps that I could complete even with fever+headache https://jitsi.github.io/handbook/docs/devops-guide/devops-guide-docker https://jitsi.github.io/handbook/docs/devops-guide/devops-gu...
- wrp 4y agoI've been using Jitsi Meet regularly for about a year. It's usually fine, but on some days I experience disconnections every several minutes.
- josteink 4y agoI’m using RNNoise as a pipewire input filter on my Linux machines, but that’s very Linux-specific and a bit “hardcore” to setup. Nice to see it getting integrated into video meeting solutions, so more people can take advantage of this awesome library.
- asicsp 4y ago>but that’s very Linux-specific and a bit “hardcore” to setup Have you tried https://github.com/noisetorch/NoiseTorch/ https://github.com/noisetorch/NoiseTorch/?
- nicolaslem 4y agoOr https://github.com/wwmm/easyeffects https://github.com/wwmm/easyeffects for noise reduction and other effects like compression and EQ for a real crooner voice in any video call application.
- kevincox 4y agoDefinitely recommend easyeffects over noisetorch. No root, high quality GUI and can work automatically in startup. I only use the noise suppression 99% of the time but having the other effects available can also be fun.
- Doman 4y agoAwesome! Could you please elaborate how to do it or post some good/not outdated links?
- josteink 4y agoDon’t remember exactly which guide I followed, but I used the build from this repo, and the instructions looks plausible: https://github.com/werman/noise-suppression-for-voice#pipewire https://github.com/werman/noise-suppression-for-voice#pipewi...
- pen2l 4y agoCreated a few years ago by Jean-Marc Valin of xiph/mozilla (who by the way is also the author of Opus codec among other things): https://gitlab.xiph.org/xiph/rnnoise/ https://gitlab.xiph.org/xiph/rnnoise/ Overview of RNNoise from the horse's mouth is here: https://jmvalin.ca/demo/rnnoise/ https://jmvalin.ca/demo/rnnoise/ Used as a Wasm module! In some ways the web is becoming more opaque. Is this the future then, a hodgepodge of binaries doing things behind the scenes? Though in this case it happens to be OSS, and it may well be a moot point -- backend is already a blackbox to the enduser, now parts of frontend are blackboxes. The practical implication is probably just that some measure of customizability is gone.
- danuker 4y agoShort of reproducible builds, you can't even check that what you're being served is, in fact, the OSS version.
- Weatherweathe 4y agoArent wasm modules still sandboxed? Reverse enginering binaries should have around same complexity than reverse enginering uglify js, not sure how they are more opaque
- pen2l 4y agoYou probably have a point but I'm thinking unuglified js code (http://www.nice2predict.org/ http://www.nice2predict.org/) is not as impenetrable as code from reverse engineered wasm binaries? The element of plausible deniability is more potent though for the nefarious actor on the other side in the case of wasm binaries.
- sabjut 4y agoIs this a troll comment? Yes, wasm works based on a compiled binary, just like any other program written in a compiled language in the past 50 years. You try to suggest that everyday users of the web are just going into the js sources of webpages and understand whats going on. With the plethora of libraries, frameworks and static optimization used in todays websites, normal people can't really dissect the inner workings of a website just by looking at the code. That's why we have tools like request analyzers etc which all would still work with compiled libraries. Compiled code has existed for half a century and we know how to work with it. Suggesting that the web is doomed because people of the future prefer rust instead of javascript is beyond any rationale.
- haunter 4y agoThis is one of the filters OBS use too (the other is Speex which is obsolete to some extent)
- atty 4y agoSomewhat tangential, but at my work we have found WebEx’s background noise removal to be absolutely amazing. So many times we’ve had someone in a meeting say “sorry about X/Y/Z, it’s so noisy”, and the rest of us won’t hear a thing. This sorta tech has gotten so good, and is a really nice quality of life improvement for remote work. (Or for meetings with people in noisy offices of course)
- geiser 4y agoSorry, but at least in my smartphone, I can understand better the unprocessed audio showcased down in the Web page, than the noise-suppresed audio. How is that?
- CharlesW 4y agoThe original audio is significantly easier to understand. This may be technically interesting, but the noise suppression is aggressive to the point that it's eating critical signal with the noise.
- SergeAx 4y agoThis is the default for online conferencing. Everyone is way better off asking other party to repeat couple of words than listening for all that noise during the whole call.
- leni536 4y agoBut this might be better served with a simplistic voice activity detection, like in mumble.
- ComputerGuru 4y ago> Everyone is way better off asking other party to repeat couple of words than listening for all that noise during the whole call. I didnt understand the first three words, for Alice it was the next two, and for Bob it was the last four. How many people are going to ask to repeat? Evolution taught us to understand over the sound of waves, crickets, rain, thunder, and more. It didn’t teach us to comprehend with half the signals masked.
- nyanpasu64 4y agohttps://jmvalin.ca/demo/rnnoise/ https://jmvalin.ca/demo/rnnoise/ says: > So what should you listen for anyway? As strange as it may sound, you should not be expecting an increase in intelligibility. Humans are so good at understanding speech in noise that an enhancement algorithm — especially one that isn't allowed to look ahead of the speech it's denoising — can only destroy information. So why are we doing this in the first place? For quality. The enhanced speech is much less annoying to listen to and likely causes less listener fatigue. > Actually, there are still a few cases where it can actually help intelligibility. The first is videoconferencing, when multiple speakers are being mixed together. For that application, noise suppression prevents the noise from all the inactive speakers from being mixed in with the active speaker, improving both quality and intelligibility. A second case is when the speech goes through a low bitrate codec. Those tend to degrade noisy speech more than clean speech, so removing the noise allows the codec to do a better job. I do think that for direct listening, the jitsi.org speech samples would be slightly more intelligible if the noise removal was tuned to pass through frequencies with mixed noise and signal. I don't know if that would be worse in a video conference. Does the speaker or listener get to choose between conservative and aggressive noise removal?
- eis 4y agoBummer, reading the title I thought Jitsi had a new de-noiser because they had RRNoise for some time. Unfortunately RRNoise has not received much advancement for a couple years. It's by now half a decade old tech. I've worked with the WASM version in the past but it can be hit or miss. Sometimes it makes the audio you want a bit weird. It also added something like 10% CPU usage and in the end we disabled it again. I'd love to see some more state of the art solution that works with WASM. Maybe even something that one could train on their own voice and filter everything else would be awesome. Because all the noise cancellation tech does not help if you sit in an environment with other people talking next to you and the AI doesn't filter it because it's voices. Sometimes coworkers use Krisp but even that proprietary paid solution is so-so.
- saghul 4y agoWhile we’ve had rnnoise integration for a while it was for “noisy environment” notifications, this is the first time we use it to actually filter audio. Also audio worklets weren’t a thing when we first introduced it. I’m not aware of any other open source (and better) models, but if any come up, we’ll certainly check them out!
- pen2l 4y agoIf you have any involvement with Jigasi or might be in the know -- are there plans to use whisper, for instance, instead of Google's API for transcription? If I recall correctly jigasi is using google's API, local transcription aligns well with the rest of Jitsi's missions.
- _joel 4y agoI prefer the sample with the noise. Seems clearer to understand
- naillo 4y agoRare to find creative real time small-weight uses of ML but I love when it's done and this has an impressive and well written explanation with it as well. Great stuff.
- mcluck 4y agoIt does seem to do a good job of eliminating noise but it seems like it gets rid of a lot of the signal too. It's much easier to understand the noisy sample than the processed one
- gnicholas 4y agoAnyone have tips for using Jitsi? I've been thinking about moving off Zoom now that they're enforcing a 40 min limit even for one-on-one calls. Does it create friction for folks who haven't used it before? Any suggested instructions to send with a meeting invite?
- e12e 4y agoWe've been using jitsi via zulip chat at work. It should be drop-in for at least small groups (one-on-one, handful of people - I have yet to investigate "conference" or "class room" size). We do unfortunately see semi-regular lock-up/freezes where one end of the stream stops for ~30 seconds. Maybe this is worse in safari vs chrome/Firefox - we have not yet experimented much with different browsers. Or maybe there's a difference between x86_64 and arm/m1/m2.
- thatsadude 4y agoAudioWorklet and WebAudio in general are buggy in Safari. - https://bugs.webkit.org/show_bug.cgi?id=244203 https://bugs.webkit.org/show_bug.cgi?id=244203 - https://bugs.webkit.org/show_bug.cgi?id=241223 https://bugs.webkit.org/show_bug.cgi?id=241223 - https://bugs.webkit.org/show_bug.cgi?id=221334 https://bugs.webkit.org/show_bug.cgi?id=221334
- e12e 4y agoThanks! I'll try and see if we fare better with Firefox.
- dividedbyzero 4y agoAs someone invited to a Jitsi meeting a while ago, not having any video background removal, a lot less audio processing and what looked like no video processing at all meant everyone was harder to understand, harder to see and any activity or clutter in the background was fully visible of course. I guess buying quality microphones and cameras for everyone involved would help. Detailed instructions are a good idea as well, I struggled a bit with the unfamiliar interface. Personally, I'd stick with the big names, long remote meetings are strenuous enough even with all the quality of life features those offer.
- hawski 4y agoIs there video conferencing software that does spatial audio for conferences? What I have in mind is that it is often problematic to understand each other while multiple people are talking. It is much easier in person. I guess it all goes down to ability to focus on directial cues of an audio source. Currently everyone are placed inside one's head so they interfere much more this way.
- gnicholas 4y agoApparently FaceTime offers this. [1] Presumably Apple will allow other companies to do it as well, since they let them offer spatial audio in other contexts. 1: https://support.apple.com/guide/iphone/change-the-audio-settings-iphb54d5dee2/ios https://support.apple.com/guide/iphone/change-the-audio-sett...
- d110af5ccf 4y agoWhy would Apple need to allow it? It's simply a matter of a given program postprocessing the various audio streams appropriately prior to muxing them for output.
- gnicholas 4y agoBecause Apple’s spatial audio varies as you rotate your head. The device knows where your head is located, but the app doesn’t (unless the OS tells it).
- rasz 4y agoYou could give up on audio portion of your current Video conferencing setup and just install Teamspeak with spatial plugin https://www.myteamspeak.com/addons/9ddfa0b2-25c2-4302-8a43-07f8819af9a9 https://www.myteamspeak.com/addons/9ddfa0b2-25c2-4302-8a43-0...
- anoa_ 4y ago
- quickthrower2 4y agoAside, Jitsi is pretty awesome for creating an video app idea quickly. The API is very easy to use.
- edlucas 4y agoSignalWire's SDK is another option and it uses an MCU instead of SFU: https://signalwire.com/products/cloud-video https://signalwire.com/products/cloud-video
- quickthrower2 4y agoWow their prices remind me of 90s phone calls
- jbaudanza 4y agoThis looks interesting. Most devices have noise suppression built in. Do you use RNNoise in tandem with the hardware noise suppression, or do you disable it? Also, are there any plans to implement this with react-native on mobile? And if so, how would you implement this, since audio worklets and WebAudio aren't available.
- perryizgr8 4y agoI haven't heard of Jitsi before this. The tech looks promising! But I can't find the link to download the software for PC/Mac, only android and ios links. Is it mobile only?
- henriquemaia 4y agoIt's a free video conference service. https://meet.jit.si/ https://meet.jit.si/
- jraph 4y agoJitsi Meet is used directly in the browser. Like at this address: https://meet.jit.si https://meet.jit.si. You can also set up your own instance. A desktop client also exists for Windows, macOS, Linux: https://github.com/jitsi/jitsi-meet-electron https://github.com/jitsi/jitsi-meet-electron - kind of not really advertised, provides remote desktop control contrary to the strictly web browser version.
- cercatrova 4y agoPersonally, I use Nvidia Broadcast which works great with noise suppression, and, best of all, works with any app because it shows up as a standard audio device. It also has camera features such as sharpening and visual noise reduction but I don't use those much. Caveat that it requires an Nvidia GPU but if you have one, it is better than any other noise suppression system I've seen.
- IronHerald 4y agoIt requires not just any Nvidia GPU but "NVIDIA GeForce RTX 2060, Quadro RTX 3000, TITAN RTX or higher"...
- pabs3 4y agoI note that RNNoise was trained on proprietary data and thus would be considered a "toxic candy" ML model under the Debian Deep Learning Team's Machine Learning Policy: https://salsa.debian.org/deeplearning-team/ml-policy https://salsa.debian.org/deeplearning-team/ml-policy