4 ms·
Well done for putting it out there! We've worked on this about 3 years ago, plus background removal (realtime alpha-matting still not really done well by anyon
by akrymski 4y ago
Well done for putting it out there!
We've worked on this about 3 years ago, plus background removal (realtime alpha-matting still not really done well by anyone), portrait re-lighting (G-Meet is now doing this) and even eye-contact (adjusting eye position to create the illusion of looking directly into the camera).
Some findings:
- Competing with dedicated H264/5 chips is very hard, especially when it comes to energy efficiency, which is something that mobile users ultimately care a lot about. Super-resolution on H264 is probably the way to go.
- It's hard to turn this into a business (corporates that pay for Zoom don't seem to care much).
PS Also a big fan of super-tiny AI models (binarized NNs, frequency-domain NNs, etc) for edge applications. Happy to chat!
- jacobgorm 4y agoThanks! Wrt the speed, I worked very long and hard on finding the right NN architecture to do this without too much overhead. My concern wrt super-resolution H264 is that you are going to have to encode and decode the full image anyway, so the cost should be very similar to doing encode-decode with network transmission in the middle. I've tried various DCT and DWT approaches, and yet not found them to be a win, but I'd be happy to learn what you guys found out. I have sent you an invite to connect at Linkedin, I am https://www.linkedin.com/in/jacob-gorm-hansen-85b724/ https://www.linkedin.com/in/jacob-gorm-hansen-85b724/ if anybody else wants to connect there.
- jamra 4y agoAre you sending occasional key frames and then just some points for a GAN to generate movement over the wire?