3 ms·

In hindsight I should probably named everything differently, but I didn't expect the software to turn into a company at some point when I initially started it o

In hindsight I should probably named everything differently, but I didn't expect the software to turn into a company at some point when I initially started it over 8 years ago for a hacker conference (see https://github.com/dividuum/info-beamer https://github.com/dividuum/info-beamer). The main challenge was to understand how to use the HEVC decoder. There's no documentation at the moment, so the only source of information is this patched FFmpeg release: https://github.com/popcornmix/FFmpeg/commits/2711 https://github.com/popcornmix/FFmpeg/commits/2711 Basically the decoder interacts with /dev/argon-hevcmem to control the HEVC hardware decoder. You let it decode into video core controlled memory and can then use the MMAL toolkit (http://www.jvcref.com/files/PI/documentation/html/ http://www.jvcref.com/files/PI/documentation/html/) to feed buffers through a scheduler and finally into the dispmanx layers handled by the hardware video scaler. That way the frames end up more or less directly on the screen. Except for the decoding part, the same pipeline of scheduler and presentation can also be used with the older H264 based encoder that's directly addressable using MMAL.

wronglebowski 7y ago
This seems like a large oversight given people usually decide to use the Pi for better documentation and support. The Pi 4 is being marketed as a "Real" desktop replacement option and I find it very surprising HEVC decoding documentation is this poor. So basically FFmpeg communicates with the HEVC hardware decoder via a bunch of rough code? Is this stable?
- dividuum 7y ago
  To be fair, these kind of low level stuff is usually handled by only a handful people, so investing a lot of time in proper documentation might be a lot of work for something useful by few users. The same is also true for DRM and in general the modern graphical APIs on stock Linux. It was surprisingly difficult to understand how all that works together. I had to rely mostly on example code. Even the man pages are incomplete. I guess it's the same situation: The kernel devs build an API, a few people (maybe even the same people) from Xorg or wayland use it and that's it. On the other hand, the initial Pi firmware for earlier models came with a bunch of bundled examples in the hello_pi directory that were incredibly useful in understanding how everything worked. (see https://github.com/raspberrypi/firmware/tree/master/opt/vc/src/hello_pi https://github.com/raspberrypi/firmware/tree/master/opt/vc/s...). Right now the forum and maybe the firmware issue tracker is the only source of information. In general, the direction of using more standardized APIs like DRM, Mesa 3D for GL and so on seems like a good idea regardless. I guess we'll just have to give all that more time. The HEVC decoder is using a patched FFmpeg with direct communication with the hardware API. In the future there will be an V4L2 API and corresponding method in FFmpeg (see https://www.raspberrypi.org/forums/viewtopic.php?t=247897 https://www.raspberrypi.org/forums/viewtopic.php?t=247897). But that's not completed yet and there's no release date announced. I would guess once that's out, the whole decoding process will be a bit cleaner than it is today. But so far I works pretty well. I ran into a ton of firmware lockups during development, but most of those are fixed now. One thing I didn't manage to do yet is to decode two videos at the same time. That's something the H264 decoder could do. See also https://www.raspberrypi.org/forums/viewtopic.php?f=70&t=252061 https://www.raspberrypi.org/forums/viewtopic.php?f=70&t=2520...

wronglebowski 7y ago

This seems like a large oversight given people usually decide to use the Pi for better documentation and support. The Pi 4 is being marketed as a "Real" desktop replacement option and I find it very surprising HEVC decoding documentation is this poor. So basically FFmpeg communicates with the HEVC hardware decoder via a bunch of rough code? Is this stable?

dividuum 7y ago
To be fair, these kind of low level stuff is usually handled by only a handful people, so investing a lot of time in proper documentation might be a lot of work for something useful by few users. The same is also true for DRM and in general the modern graphical APIs on stock Linux. It was surprisingly difficult to understand how all that works together. I had to rely mostly on example code. Even the man pages are incomplete. I guess it's the same situation: The kernel devs build an API, a few people (maybe even the same people) from Xorg or wayland use it and that's it. On the other hand, the initial Pi firmware for earlier models came with a bunch of bundled examples in the hello_pi directory that were incredibly useful in understanding how everything worked. (see https://github.com/raspberrypi/firmware/tree/master/opt/vc/src/hello_pi https://github.com/raspberrypi/firmware/tree/master/opt/vc/s...). Right now the forum and maybe the firmware issue tracker is the only source of information. In general, the direction of using more standardized APIs like DRM, Mesa 3D for GL and so on seems like a good idea regardless. I guess we'll just have to give all that more time. The HEVC decoder is using a patched FFmpeg with direct communication with the hardware API. In the future there will be an V4L2 API and corresponding method in FFmpeg (see https://www.raspberrypi.org/forums/viewtopic.php?t=247897 https://www.raspberrypi.org/forums/viewtopic.php?t=247897). But that's not completed yet and there's no release date announced. I would guess once that's out, the whole decoding process will be a bit cleaner than it is today. But so far I works pretty well. I ran into a ton of firmware lockups during development, but most of those are fixed now. One thing I didn't manage to do yet is to decode two videos at the same time. That's something the H264 decoder could do. See also https://www.raspberrypi.org/forums/viewtopic.php?f=70&t=252061 https://www.raspberrypi.org/forums/viewtopic.php?f=70&t=2520...