12 ms·
Nvidia Broadcast App
- suyash 6y agoAnything similar like this exists for Mac?
- godelski 6y agoOr linux?
- jcastro 6y agoCheck out https://github.com/lawl/NoiseTorch https://github.com/lawl/NoiseTorch
- rahimnathwani 6y agoFor virtual background without green screen: mmhmm
- aantix 6y agoKrisp - Mute background noise in any communication app https://krisp.ai/ https://krisp.ai/
- zaroth 6y agoThe Broadcast app offers three AI-powered features: Noise Removal: remove background noise from your microphone feed – ... Virtual Background: remove the background of your webcam feed and replace it with game footage, a replacement image, or even a subtle blur. Auto Frame: zooms in on you and uses AI to track your head movements, keeping you at the center of the action even as you shift from side to side... These features can be used beyond game broadcasting as well — from video conferencing at home with Zoom, to gaming with friends on Discord.
- Ecco 6y agoNoise reduction, background removal. Technologies that have been around for ages. But now apparently it's "with the power of AI" so it's newsworthy?
- dgellow 6y agoAI just means deeplearning here. The past few years have seen great improvement when applying deep learning to a bunch of already established domains, such as noise reduction and background removal
- bayesianbot 6y agoWhat other software would you suggest for good quality real time background removal? The ones I've tried I've been stuck with pretty bad artefacts.
- harvey9 6y agoI pinned a piece of green cloth behind me and used OBS with the virtual webcam plugin. Had someone in a meeting comment that there's no artifacts on my feed. He thought it was because I have a shaved head though.
- ryneandal 6y agoDepends on where this product ends up, I suppose. If it learns and improves it's current performance with my admittedly noisy background, then the improvements on the technology would be pretty incredible. I mean, isn't the whole point of the tech industry to iterate on old tech and make life better with each improvement?
- resoluteteeth 6y ago> For those of you that want to try out the AI noise removal capabilities but aren’t ready to upgrade to an RTX GPU yet, we have also patched RTX Voice with support for NVIDIA GeForce GTX GPUs. Though, of course, your mileage may vary on older cards. That's a strange way to say "yeah we admit that we were just blocking older GPUs in the installer and you guys noticed so decided to stop."
- motoboi 6y agoWell it kinds looks like your car manufacturer stating in the manual that your car cannot tow. You know it just a matter of installing one, but they basically don't want to spend time supporting something they have no interest in.
- bluGill 6y agoIn the case of cars often a car can tow, but it shouldn't. I know someone who burned out a transmission because the trailer was overweight (in this case it was a truck that the manual said max tow weight was 1/4th what he was towing). I haven't seen it myself, but my EMT friends have had to respond to cases where someone was towing I work for John Deere, it is well known that ECU code can change the horse power of our tractors. What isn't as well known is the tractors with higher horsepower ECUs have more warranty work, and that is a case where we engineer the system for the change (there is more than an ECU changed in getting higher horsepower if you buy it that way from us because we don't want warranty claims) If you know what the real limits of your system are you will sometimes discover the stated limits are less than the real limits. However other times the stated limits are there for a reason.
- andybak 6y agoGive people the facts. Let them decide but make sure you're protected from supporting idiots. Better than coming out of it looking like the evil dudes. EDIT being most well known outside your sector for suing your customers is generally a bad look
- ryneandal 6y agoDue to having to keep an eye on my kids during virtual school, I'm stuck in the family room which is incredibly noisy with toys and furniture. I've played with the software for about 5 minutes now and think the virtual background functionality works incredibly well: https://imgur.com/a/0ByEYF9 https://imgur.com/a/0ByEYF9
- codezero 6y agoEspecially given that you have a bright background light, also that it cut through the space between your headphones. Pretty neat, and better than zoom’s silhouette guess.
- ryneandal 6y agoYeah, no issues with the shades drawn on the window, it's impressive. I hadn't noticed the space between my head and headphones prior to this comment, great catch. I'll definitely be using it during work meetings now, I'm sold.
- localhost 6y agoI'm running build 460 of the NVIDIA drivers on Win 10 because I'm using the DirectML support for WSL 2. It appears that Broadcast doesn't work with this as it tells me "cannot start service" each time I try to launch the Broadcast app. Has anyone else gotten this to work, and what build of the NVIDIA drivers are you using on Windows?
- cptskippy 6y agoI just updated my Quadro RTX 4000 to the latest available driver and it's only R450, when I try to install Broadcast it says I need R455 or newer.
- babypuncher 6y agoCheck for beta drivers. If I recall, Nvidia only cuts new "stable" Quadro drivers very rarely, putting them through very rigorous validation since they are pro-level hardware.
- cptskippy 6y agoThe latest driver was released 8/18/2020 and Broadcast's system requirements say they're supported... There don't appear to be beta drivers.
- brainless 6y agoHere is the video explaining it nicely with demos: https://www.youtube.com/watch?v=GRFjfGH87Dk https://www.youtube.com/watch?v=GRFjfGH87Dk
- syntaxing 6y agoYou should see what Nvidia does for CAD software...They used to force you to use Quadro cards only (which is maybe 2X-3X price of their consumer version) even though the GT/X series worked just as good. People figured out how to change the registry setting to make it work until they finally caved and supported it "somewhat" (where if it breaks it breaks and this was maybe 5 years after the registry hacking). Nvidia has always been notorious for making people use the "good stuff" (expensive) even though it's not really required.
- fock 6y agoor for running their GPUs in VMs...
- LeSaucy 6y agoIs that not a thing now?
- moojd 6y agoIt looks like they are still disabling it: https://youtu.be/AG_ZHi3tuyk?t=627 https://youtu.be/AG_ZHi3tuyk?t=627
- fock 6y agowhich ist completely user hostile, as the original idea (back with certified Z800s, RHEL5 and FX3800s) of a dual-OS-workstations has not panned out. Today it's basically nagging developers and technical users, who don't want to run proprietary software unsandboxed (but want CUDA...) and have a hassle-free experience into buying quadros. I can't think of any business-case for that, because cracking down at the shady ML-cloud-vendors works out okish right now for them and at unis people won't notice at large...
- mey 6y agoWith Ryzen's support of ECC, if AMD brought SR-IOV to its non-work station radeon cards, it would be another interesting feather in its cap. They do have product announcements coming up, bought doubt that is an important feature to most users.
- tomasreimers 6y agoInteresting, I wonder if on a meta-note we're seeing an evolution of Nvidia's strategy. For a while, it was hardware, then it was leveraging that hardware to create developer-facing APIs (CUDA being a driver for many ppl in ML to buy Nvidia in particular), and now it seems they may be trying out leveraging that compute to create specialized, end-user facing services. Are there any more examples of this? I wonder if Nvidia will try to move upmarket to own the best UI powered services (e.g. if you want a camera with a virtual background, it has to be nvidia or something similar)
- 01100011 6y agoI can't seem to find a single page that lists all of Nvidia's current vertical software stacks, but they are legion. Aerial(5G), Metropolis(smart cities), Drive(self driving), RAPIDS(big data?), Clara(genomics), IndeX(visualization), Isaac(robotics)... etc.. etc... They basically have to move towards vertical software stacks requiring their HW. GPUs are becoming commoditized, with integrated GPUs from AMD and Intel becoming good enough to play modern games now. 8k(haha) may keep things interesting for a while, but eventually AMD/Intel will compete there as well. I don't know about user-facing apps though. So far, most of what they've released(i.e. Gaugan) seem more like demos.
- person_of_color 6y agoThey are becoming the Apple of B2B compute. And Shield is their killer consumer app. Watch out for FAANNG
- madeofpalk 6y agoI don't think it's particurly interesting or novel - they're just creating a complete ecosystem to keep customers within so they keep buying their hardware. It's not exactly vendor lockin, but it's that same vibe.
- jeena 6y agoOnly a windows version, meh.
- redisman 6y agoHow mature is this software? I tried to use RTX voice on an important call and it was not working at all causing me to be late and left a bad taste for these for serious use.
- ShamelessC 6y agoI've had no issues with it. The real problem was Windows 10's extremely frustrating and confusing audio engine.
- sinak 6y agoFor anyone not using an nVidia graphics card, I highly recommend https://krisp.ai/ https://krisp.ai/ for the voice cleanup part. Works wonders.
- nerdbaggy 6y agoDo you know of any other companies offering a product like krisp? They seem to be one of the only players.
- loa_in_ 6y agoAdmittedly, it's a young branch, no wonder there's few players to date
- emcq 6y agoBabbleLabs is the closest but they got acquired by Cisco. Google Meets also has similar tech https://support.google.com/meet/answer/9919960?hl=en https://support.google.com/meet/answer/9919960?hl=en Audatic was really cool but they got acquired by a hearing aid company. Whisper.ai has very similar processing but they are also focused on hearing aids.
- tsumnia 6y agoOverall I'm very impressed and plan to incorporate the Background Blur into my lecture videos (since I had to create a 'decorative' background for them anyway). So everyone can see how it looks with a makeshift lighting setup (kitchen lights, overhead lights, and a 3 bulb lamp). Background Removal: https://i.imgur.com/e3y4kwX.png https://i.imgur.com/e3y4kwX.png Background Replacement: https://i.imgur.com/AajgJIf.png https://i.imgur.com/AajgJIf.png Background Blur - Low Setting: https://i.imgur.com/CeXmneX.png https://i.imgur.com/CeXmneX.png Background Blur - Max Setting: https://i.imgur.com/OoJ76ow.png https://i.imgur.com/OoJ76ow.png Frame Tracking - https://i.imgur.com/wQ8IOq1.mp4 https://i.imgur.com/wQ8IOq1.mp4 Full Album: https://imgur.com/a/RWECpd4 https://imgur.com/a/RWECpd4 EDIT: As a followup to a question my friend asked - memory wise it is sitting idly at 105MB; running OBS it went to 670MB. EDIT2: The blur effect holds even if you move to the background as well without sharpening the objects in the background. It did look a little "off" but that's mostly because a sharply rendered body interacting with a blurred environment looks off.
- iseanstevens 6y agoNice smile :)
- tsumnia 6y agoGotta look good for all those movie producers that just found their next big star :D
- strombofulous 6y agoCan you post an example of your edit 2? Seems like it'd be interesting to watch
- movedx 6y agoI did a quick demo here: https://www.youtube.com/watch?v=aoa0aB1AXtI https://www.youtube.com/watch?v=aoa0aB1AXtI Hardware: Sony A6000 Mirrorless camera; 2x Elgato Key Lights for illumination; GameCapture HD60 S; RTX 2080 Super; OBS for the video feed and recording.
- wiradikusuma 6y agoI have Sony A6000 with Elgato too. Do you mind showing your setup esp. camera placement? I'm curious your eyes can look at the audience while in front of PC.
- Godel_unicode 6y agoBy looking at the camera while recording, watch the first few seconds again.
- nl 6y agoJust noting that the Turing series of cards and later (which isn't just RTX, but also the GTX16* the the T4) does have specialised hardware which helps with the video tracking side of this. They have hardware optical flow[1] which doesn't seem widely known, but would absolutely help with parts of this - especially the reframing. [1] https://developer.nvidia.com/opticalflow-sdk https://developer.nvidia.com/opticalflow-sdk
- gavinray 6y agoOdds that this winds up with a Linux release eventually? =(
- Godel_unicode 6y agoI'd say the chance is roughly the same as the percentage of Nvidia customers running desktop Linux, I imagine. The three of you must be used to this by now though?
- gavinray 6y agoMy RTX-2060 runs well on Pop_OS!/Ubuntu I've had pretty much nothing but good experiences running Nvidia on Linux the past 4-5 years, barring one strange issue with multiple monitors, a DisplayLink adapter, and trying to rotate a screen sideways. I can see fiscally how it doesn't make a lot of sense for them -- but why write a platform-dependent implementation and not use code that's generic? Is the problem the GUI? Build it with Qt or something
- Dahoon 6y agoAh yet another Nvidia thing that would work everywhere but is blocked in software because greed. Remember how raytracing was supposed to only work properly on RTX but when people got mad suddenly it wasn't only RTX cards? And now we are even seeing it on the old and slow PS4. Next they'd likely do something crazy like require unnecessary hardware in screens or something. Oh wait..
- raobit 6y agoI have 2 Q's 1. As in video on background removal , why was the chair not blurred and because generally in background removal only the person is visible and all rest are blurred, how is it working? 2. How Background Noise removal works, i mean how it is so smoothly removing background noise.How it is able to distinguish which is the primary and secondary,i know it's AI but still want to know the logic and basic understanding.Thanks
- suff 6y agogreat, I'll just run it on this hardware you 'released' today... </sarcasm>
- brian_herman 6y agoThis seems better than RTX voice I have my ears blown out when someone uses it on discord and it crashes.