4 ms·
Since H.264 has P-frames and B-frames and so on which I think would be by definition stateful, I assume that processing is just pushed out to the client/userspa
by zerocrates 6y ago
Since H.264 has P-frames and B-frames and so on which I think would be by definition stateful, I assume that processing is just pushed out to the client/userspace somehow?
- stormer2000 6y agoStateless refer to the HW Interface. The stateless HW can accept decoding jobs in any order as long as all the information is properly provided (parameters extracted and deduces from the bitstream along with the previously decoded references). As a side effect, it is trivial to multiplex multiple streams using this type of HW. The V4L2 layer keeps a bit of the state (more like caching, to avoid re-uploading too much information for each jobs). The userspace is responsible for bitstream parsing and DPB management (including re-ordering).
- zerocrates 6y agoOh so, you just provide whatever reference frames, if any, are needed, and it's just on you to make sure you've decoded what's necessary first? The difference here basically being that the hardware will not do the "bookkeeping"?
- stormer2000 6y agoCorrect.
- zerocrates 6y agoThanks for explaining.
- vlovich123 6y agoAre there performance implications of needing to upload the entire state needed for a single frame? Or do none of these encoders have such caching anyway & thus it's just pushing the complex pieces of resource management out to user space where it belongs better?
- megous 6y agoI don't think anything is uploaded anywhere, you just need more RAM to keep frames around as long as they are necessary. The decoder operates on data in system's RAM.
- vlovich123 6y agoIs that generally true to be faster rather than having dedicated RAM alongside the ASIC? Or are the unit economics not worth it and generally unified memory systems is the current dominating design?
- londons_explore 6y agoConsidering most pixels in the reference frames will be read, on average, less than once per generated frame, it makes no sense to have dedicated RAM.
- magicalhippo 6y agoBut each reference frame is on average used for many generated frames, no? I mean that's kinda the point of them, isn't it?
- megous 6y agoInput is way smaller than output, so memory performance considerations there probably don't even register in the larger scale of things. (having to write the decompressed frame to RAM and read it again to scan it out to display) Say 4-8kiB per frame on input leads to 4MiB frame on output.
- magicalhippo 6y agoI admit I'm not familiar with h246, I thought the motion vectors and such was applied on the decompressed reference image. At least that's how we implemented the psedudo-MPEG1 encoder/decoder in class. Not having to decompress the reference frame for every decoded frame seems like a win.
- bob1029 6y agoI believe this is still an interframe approach, but something regarding the actual underlying code is different than a traditional implementation. From an information theory perspective, you absolutely must have some way to pass intermediate frames around, otherwise you are just talking about some variant of an intraframe technique and all of the video efficiency losses that would go along with that (i.e. encoding each video frame as JPEG).
- stormer2000 6y agoAll H.264 (and other CODECs) prediction are covered. As stated correctly in previous comment, userspace do the bookkeeping. Kernel is aware of the reference buffers and it's attachements (HW specific buffers). All references needed for predictive decoding (regardless if it's B or P), are programmed for each jobs. Exception to FMO and ASO mode, which is rarely supported in HW, even FFMPEG sw decoder didn't bother implementing that
- acchow 6y agoShould be the reverse - these are pulled into behind the interface so you don't need to deal with them in userspace.