4 ms·
I've worked on a IPTV broadcasting system and this isn't as obvious as you'd think. The big issue here is quality - most hardware encoders hugely lag behind so
by izacus 6y ago
I've worked on a IPTV broadcasting system and this isn't as obvious as you'd think.
The big issue here is quality - most hardware encoders hugely lag behind software encoders. By quality I mean visual quality per bytes/s. Which means that using a quality software encoder like x264 will save you massive amount of money in bandwidth costs because you can simply go significantly lower in bitrate than you can with a hardware encoding block.
At the time, our comparisons showed that you could get away with as low as 1.2MBps for 720p stream where with an enterprise HW encoder you'd have to do about 2-3MBps to have the same picture quality.
That's one consideration. The other consideration is density - at the time most hardware encoders could do up to about 4 streams per 1U rack unit. Those rack units cost about half the price of a fully loaded 24+ core server. Even GPUs like nVidia at the time could do at most 2 encoding sessions with any kind of performance. On CPU, we could encode 720p on about 2 Xeon cores which means that a fully loaded server box with 36+ cores could easily do 15-20 sessions of SD and HD and we could scale the load as necessary.
And the last consideration was price - all HW encoders were significantly more expensive than buying large core count rack-mount servers. Funny enough, many of those "HW specialised encoding" boxes were running x86 cores internally too so they weren't even more power efficient.
So in the end the calculation was simple - software encodes saved a ton of money on bandwidth, it allowed better quality product because we could deliver high quality video to people with poor internet connectivity, it made procuring hardware simple, it made the solution more scalable and all that at the cost of some power consumption. Easy trade. Of course the computation is a bit different with modern formats like VP9 and H.265/HEVC - the software encoders are still very CPU intensive so it might make sense to buy cards these days.
Of course, we weren't Google and couldn't design and manufacture our own hardware. But seeing the list of codecs YouTube uses, there's also one more consideration: flexibility. HW encoding blocks are usually very limited at what they can do - most of them will do H.264, some of them will stretch to H.265 and maaaaaybe VP9. CPUs will encode into everything. Even when a new format is needed, you just deploy new software, not a whole chip.
- jng 6y agoVery interesting description. Are you familiar at all with the details of FPGAs for these very same tasks, especially the EV family of Xilinx Zynq Ultrascale+ MPSoC? They include hardened video codec units, but I don't know how they compare quality/performance-wise. Thanks!
- izacus 6y agoI'm afraid I don't have any experience with those devices. Most HW encoders however struggle with one thing - the fact that encoding is very costly when it comes to memory bandwidth. The most important performance/quality related process in encoding is having the encoder take each block (piece) of previous frame and scan the current frame to see whether it still exists and where it moved. The larger area the codec scans, the more likely it'll find the area where the piece of image moved to. This allows it to write just a motion vector instead of actually encoding image data. This process is hugely memory bandwidth intensive and most HW encoders severely limit the area each thread can access to keep memory bandwidth costs down and performance up. This is also a fundamental limitation for CUDA/gpGPU encoders, where you're also facing a huge performance loss if there's too much memory accessed by each thread. Most "realtime" encoders severely limit the macroblock scan area because of how expensive it is - which also makes them significantly less efficient. I don't see FPGAs really solving this issue - I'd bet more on Intel/nVidia encoding blocks paired with copious amount of onboard memory. I heard Ampere nVidia encoding blocks are good (although they can only handle a few streams).
- spuz 6y agoThat is interesting context for this quote from the article: > "each encoder core can encode 2160p in realtime, up to 60 FPS (frames per second) using three reference frames." Apparently reference frames are the frames that a codec scans for similarity in the next frame to be encoded. If it really is that expensive to reference a single frame then it puts into perspective how effective this VPU hardware must be to be able to do 3 reference frames of 4K at 60 fps.
- daniellarusso 6y agoI always thought of reference frames as like the sampling rate, so in that sense, is it how few reference frames can it get away with, without being noticeable? Would that also depend on the content? Aren’t panning shots more difficult to encode?
- 6y ago
- colonwqbang 6y agoMy work is in IP cameras so I'm aware of these tradeoffs. I guess what I didn't expect is that Google could design their own encoder IP to beat the current offerings with a big factor at the task of general video coding. I guessed that Google actually built an ASIC with customised IPs from some other vendor. But maybe Google did do just that?