7 ms·
Cisco Leap Frogs H.264 Video Collaboration with Real-Time AV1 Codec
- kimburgess 7y agoTalk here: https://www.youtube.com/watch?v=Op-rzboJ_1Q https://www.youtube.com/watch?v=Op-rzboJ_1Q
- ragerino 7y agoGreat to see improvements on the AV1 encoding side. I remembered reading something similar coming from Europe -> Allegro DVT AL-E210. http://www.allegrodvt.com/products/silicon-ips/al-e210/ http://www.allegrodvt.com/products/silicon-ips/al-e210/ How does your solution compare to Allegros one?
- snops 7y agoCisco are not using dedicated hardware logic like Allegro DVT I think. For phones etc, dedicated hardware will be very useful, but it takes time to get into products, which is why a real time CPU encoder is very interesting. Looking at the video (https://vimeo.com/344366650 https://vimeo.com/344366650) of the Cisco talk, at 20m18s they say a "real time low latency HD software encoder" [emphasis added], which confirms it.
- chubs 7y agoCoincidentally i'm trying out rav1e today (the open-source command line av1 encoder). At 1080p for a 2m40s movie trailer on a current-model mac mini 6-core, it's running at 0.153FPS. That means it's running at 157x slower than real-time. Kudos to these cisco chaps for being roughly 157x faster than rav1e!
- benj111 7y ago....Or having a PC roughly 157x faster.
- fluffything 7y agoYou should open an issue, the fix might be as simple as changing your build flags.
- ksec 7y agoYou simply turn off all the tools that are slow, and you end up with an AV1 encode that is very fast, but quality wont be anywhere close to your normal AV1 encode. I don't doubt it will be of higher quality than AVC. After all AVC was the work started pre 2000 and published in 2003.
- mjevans 7y agoTwo important things it will be: 1) Royalty (and annoying patent) free. 2) A reason for hardware decoders to exist.
- derf_ 7y agoIt is easy to dismiss what Cisco demonstrated like this, but if you listen to the talk, they argue that having more tools available gives you more room to optimize for quality per bit per cycle, in the same way that having more choices allows you to optimize better for just quality per bit (or any other objective, really). So it's not just a matter of turning things off, but of making good choices. It is also not just beating AVC. They claim that it is higher quality than their own HEVC work, which they invested in heavily before it become clear that the licensing situation was not going to just "work itself out". They once told us that even they were surprised how many of the HEVC tools you could wind up using for real time, if you used them judiciously (basically: all of them). I have worked with some of the team responsible for this project before, and we collaborated with them on some AV1 tools (most notably the CDEF loop filter, but also entropy coding and a few other things). They know their stuff, and really do seem to be leaps and bounds ahead of everyone else on the RTC side of things.
- theandrewbailey 7y agoI encoded about 100 1080p-sized images to AVIF yesterday. I was a little disappointed that each of them were taking about 30 seconds, fully loading an i7-2600 CPU. Realtime AV1 encoding is good news, maybe those optimizations can reach the still image format?
- metildaa 7y agoYour CPU is missing AVX2 & AVX512 (and numerous other CPU extensions), the amount of optimiztion that can happen for your CPU is limited. https://en.m.wikipedia.org/wiki/Advanced_Vector_Extensions https://en.m.wikipedia.org/wiki/Advanced_Vector_Extensions
- Causality1 7y agoWhen can we expect to see hardware AV1 encoding/decoding in mobile SoCs?
- clouddrover 7y agoIt's starting to happen. Here's a SoC announcement from Realtek for AV1 decoding: https://www.realtek.com/en/press-room/news-releases/item/realtek-launches-worldwide-first-4k-uhd-set-top-box-soc-rtd1311-integrating-av1-video-decoder-and-multiple-cas-functions https://www.realtek.com/en/press-room/news-releases/item/rea...
- ksec 7y ago>hardware AV1 encoding/decoding in mobile SoCs?
- arghwhat 7y ago"Mobile" is subjective. There are laptops with desktop chipsets after all... Who stops a phone from having a set-top box chip? :)
- errantspark 7y agoFair enough, but those laptops are meant to be used tethered to a power source. Set top box chips are generally far more power hungry than their mobile counterparts, just like desktop chips vs laptop chips. The issue here is that I'm not sure there's much of a market for phones that don't work on battery for more than 15 minutes. Then again, there's a market for water-cooling blocks for phones so I guess anything is possible.
- arghwhat 7y agoThose laptops do come with a battery, though, so someone out there is enjoying a whole 22 minutes of battery capacity. In all seriousness, though, integration into a set top box chip is a sign of commodity: A low power chip with limited application, where it is implemented not to power a large marketing department, but simply out of potential necessity. This is to me a far bigger indicator of "things coming soon" than, say, integration into a high-end GPU.
- stefan_ 7y agoReal-time is such an unfortunate term when talking about video collaboration. A codec that can achieve real-time encoding speed is still useless if it has a 10 frame latency to output the first data, and arguably not real-time.
- morerunes 7y ago~1/3 second is perfectly acceptable to me to be within the range of "real time", and is quick enough that I'd be willing to bet most people won't notice even in a two way conversation.
- Benjamin_Dobell 7y agoFor live streaming and chat that's fairly acceptable. However, don't forget there are other potential use cases, particularly since cloud gaming is supposedly becoming a thing again.
- spyder 7y ago... and on what kind of hardware, because it's not so much useful if you need a supercomputer for it.
- MisterTea 7y agoThe proper term should is low-latency. Real-time implies deterministic timing of events which must meet a deadline. Real-time can give you low latency but so can a system with a high enough throughput.
- kalleboo 7y agoSo how far away is it until my phone can record 4K @ 60fps like it can with HEVC? I'm guessing we'll need hardware support built on these optimizations?
- slimscsi 7y agoMy guess, 5 years. And the cisco “optimizations” May work well for a conference call with a perfectly still camera and low spacial and temporal information density. Applying the same techniques to handheld action video would produce very poor results.
- CWuestefeld 7y agoIn my experience, the weakness of current conferencing systems isn't in the software - it's a hardware problem. Specifically, their ability to capture and transmit usable speech is really poor. Between echo and poor noise gating, I'm not exaggerating when I say that I can make out fewer than 1/3 of the words in my team's conference calls. I don't care how efficient the video codec is, if I can't clearly understand what the other people are saying, it's all useless.
- raisedbyninjas 7y agoI think the latency is the killer for our remote conference calls. So much extra delay and effort goes into not talking over each other that isn't an issue in a live meeting.
- metildaa 7y agoSounds like your conferencing system has large jitter buffers. If your internet is stable, you should be able to reduce the buffer. To test your network, go to https://www.dslreports.com/speedtest https://www.dslreports.com/speedtest and see if you have significant jitter or bufferbloat. If your rating is an A, then your good to go, otherwise its time to clean up your local network :P
- holy_city 7y agoEcho cancellation and voice activity detection (VAD/what they use for gating), AGC, etc is all done in software in modern VOIP (or firmware? It's on a dedicated processor). And it's quite good if you spend the money on it. Those cheap Cisco conference phones do sound terrible, and I don't think people replace them very often.
- metildaa 7y agoMoney isn't a fix all for AEC, frankly Google's WebRTC AEC is the best commercially available AEC. You can see it in action when on a call over Signal Private Messenger, it knocks out background noise (eg: loud car noises if your walking down the street) like a champ.
- p0nce 7y agoWell, real-time 4kp60 HEVC encoding is a reality since 2014 https://www.businesswire.com/news/home/20140904005279/en https://www.businesswire.com/news/home/20140904005279/en Since that time HEVC encoding has only become faster and less expensive. This real-time AV1 encoder achieves 1080p30, which is simply speaking 8 times less pixels than that 2014 demo. And compares against H.264 which says it all.
- clouddrover 7y ago> And compares against H.264 which says it all They compared against HEVC as well. To quote the article: "This means that we can substantially raise quality, while saving bits, all with a very usable CPU footprint. We have found that the real-world compression/speed trade-offs for AV1 are in fact excellent, and better than HEVC." And the business problem with HEVC is: "HEVC (aka H.265) comes with unacceptable patent cost, risk and uncertainty."
- p0nce 7y agoIt hasn't stopped HEVC from winning https://www.youtube.com/watch?v=vgE8-4rcXl0 https://www.youtube.com/watch?v=vgE8-4rcXl0
- clouddrover 7y agoWhat do you believe HEVC is winning? The major online video platforms (YouTube, Vimeo, Netflix, Twitch, etc.) are all going to AV1. HEVC had its shot but the terrible licensing stunted its growth and AV1 will replace it. Leonardo Chiariglione says the MPEG business model is broken and I think he's right: http://blog.chiariglione.org/a-crisis-the-causes-and-a-solution/ http://blog.chiariglione.org/a-crisis-the-causes-and-a-solut...
- p0nce 7y agoLeonardo Chiariglione also said "AOM will certainly give much needed stability to the video codec market but this will come at the cost of reduced if not entirely halted technical progress. " which is a very real concern. (a bit tired arguing with people defending Google's contraption vs a codec with innovation from dozens of companies)
- thruhiker 7y agoI haven't used Webex or any Cisco conferencing offerings recently but I have been very impressed by Zoom. It works really well when using computer audio with AirPods. I've used this setup for meetings across continents without thought to latency. This may be a result of Internet connections improving in general. I found video conferencing to be distracting in the past but surprisingly good now.