3 ms·

Ok, a simpler library on top of, and in collaboration with, the existing library I can get behind. Perhaps they should add this to their FAQ.

HarHarVeryFunny 4y ago
From the project documentation there's no indication of it being based "on top of" V4L2, in fact the exact opposite. libcamera has compatability layers that add V4L2 and "Android Camera HAL v3" (which I'm not familiar with) on top of libcamera. https://libcamera.org/docs.html https://libcamera.org/docs.html
- mort96 4y ago
  A lot of software uses the video4linux kernel interface directly. That v4l2 compatibility layer is to make those applications go through libcamera even though they think they're using the kernel interface directly.
  - HarHarVeryFunny 4y ago
    Exactly - that's the definition of a compatibility layer! If you need to use a camera that's supported by libcamera, but not natively by v4l2, and prefer to use the v4l2 API, then libcamera's v4l2 compatability layer will let you do that. But it seems the goals of libcamera are quite different than v4l2. v4l2 (which I used to use years ago) seems more about supporting the minimum common feature set of cameras - basic video streaming, while libcamera appears more about supporting the greatest common feature set - with CPU implementation of features where necessary (hence the Mesa comparison).
    - mort96 4y ago
      The thing is, libcamera uses video4linux, because video4linux (together with the rest of the media infrastructure userspace API[1]) is how camera devices are exposed by the kernel. It's just that not all cameras are as simple as "find the /dev/videoX device and throw ioctls at it" anymore; you need to set up the configurable subdevice node graph (also using video4linux APIs), and maybe the frames the device gives you need post-processing, etc. There's nothing in libcamera which you couldn't do with video4linux APIs + userspace code, since libcamera is itself just something which sits in userspace and uses video4linux APIs. The compatibility layer simply makes applications which use video4linux in the "find the /dev/videoX device and throw ioctls at it" way work even with cameras which are much more complicated to set up and configure. But it just replaces one usage pattern of the video4linux kernel interface with a different, more complex usage pattern of the video4linux kernel interface (and the media controller kernel interface) + potential userspace processing. So libcamera is absolutely "on top of" video4linux is my point. [1] https://www.kernel.org/doc/html/latest/userspace-api/media/index.html https://www.kernel.org/doc/html/latest/userspace-api/media/i...
      - HarHarVeryFunny 4y ago
        OK I'll take your word for it (libcamera using v4l2), but are you sure there arn't also any cameras supported by libcamera directly? There must be some reason for libcamera's v4l2 compatability layer ... I hadn't heard of libcamera until reading this story/thread, and it seems to provide useful functionality, so I'm trying to understand the amount of apparent hostility there seems to be towards it.. saying it should be part of v4l2. It certainly seems to provide added functionality "over and above" v4l2, and makes sense if it uses v4l2 for lower level camera access, although from a user perspective that really makes no difference.
        mort96 4y ago
        To answer the first part: There's a lot of software out there which iterates through `/dev/videoX` devices, opens one of them (maybe by presenting the user with some UI to choose between them), and then just starts using it. You can query which pixel formats are supported and set one, you can query the range of supported frame rates and set a frame rate, you can query and set resolutions, etc. by interacting only with the /dev/videoX device you chose. However, this only works with some cameras; notably, it works with USB webcameras and most laptop webcameras. But in the embedded/phone world, and with some recent laptops, this can't work. There is simply no single device which is "the" camera. The camera system is a complex pipeline of different image processing nodes. Here's how the graph looks on the hardware I usually work with: https://i.imgur.com/NSmu4Tj.png https://i.imgur.com/NSmu4Tj.png. The actual camera is the ov5645 node at the top, but that's not really useful in itself. You need to configure the media graph; in this case, I have set it up as ov5645 -> csiphy1 -> csid1 -> ispif1 -> vfe0_pix, and finally, /dev/video3 is a video4linux device (as opposed to a "subdev", which the others are) which an application can interact with using the video4linux IOCTLs. And importantly, you can't configure things like resolution, cropping, etc. on /dev/video3; you have to use the media control API to configure the ov5645 node's resolution, then you can additionally configure scaling and cropping and other forms of image processing on the vfe0_pix node. If you want to access your frames without doing any processing, you can instead link up the ispif1 node to a vfe0_rdiX node (RDI = Raw Dump Interface). My graph is relatively simple because only the vfe0_pix node does actual configurable image processing, but there's no reason the hardware couldn't be structured in a way where different nodes do different kinds of processing. This is the stuff libcamera understands, and the stuff which normal "open a /dev/videoX device and throw ioctls at it" style application doesn't understand. The goal is for libcamera to figure out what graph it has to build to do what you want, to figure out what image processing operations are available at your various nodes, maybe apply various post-processing steps such as debayering if that's not handled by the image processing hardware, etc. So the value of the compatibility layer is that an application which expects to just care about a /dev/videoX device can have its API calls intercepted, and libcamera will basically try to make it look to the application as if it's talking to a simple device where framerate/resolution/pixel format control/post processing/etc is all handled by the /dev/videoX device. In the background, libcamera will build the media graph, configure the graph nodes, do post processing, and whatever else is necessary to make the camera work. It basically makes a complex camera system look to the application as if it is a simple USB webcam. I hope some of this makes the problem that's being solved here a bit more clear. As to why people are hostile to it, I'd just chock that up to the general culture in Hacker News being of making fun of and demeaning other people's work, especially when they don't understand it.

HarHarVeryFunny 4y ago

From the project documentation there's no indication of it being based "on top of" V4L2, in fact the exact opposite. libcamera has compatability layers that add V4L2 and "Android Camera HAL v3" (which I'm not familiar with) on top of libcamera. https://libcamera.org/docs.html https://libcamera.org/docs.html

mort96 4y ago
A lot of software uses the video4linux kernel interface directly. That v4l2 compatibility layer is to make those applications go through libcamera even though they think they're using the kernel interface directly.
- HarHarVeryFunny 4y ago
  Exactly - that's the definition of a compatibility layer! If you need to use a camera that's supported by libcamera, but not natively by v4l2, and prefer to use the v4l2 API, then libcamera's v4l2 compatability layer will let you do that. But it seems the goals of libcamera are quite different than v4l2. v4l2 (which I used to use years ago) seems more about supporting the minimum common feature set of cameras - basic video streaming, while libcamera appears more about supporting the greatest common feature set - with CPU implementation of features where necessary (hence the Mesa comparison).
  - mort96 4y ago
    The thing is, libcamera uses video4linux, because video4linux (together with the rest of the media infrastructure userspace API[1]) is how camera devices are exposed by the kernel. It's just that not all cameras are as simple as "find the /dev/videoX device and throw ioctls at it" anymore; you need to set up the configurable subdevice node graph (also using video4linux APIs), and maybe the frames the device gives you need post-processing, etc. There's nothing in libcamera which you couldn't do with video4linux APIs + userspace code, since libcamera is itself just something which sits in userspace and uses video4linux APIs. The compatibility layer simply makes applications which use video4linux in the "find the /dev/videoX device and throw ioctls at it" way work even with cameras which are much more complicated to set up and configure. But it just replaces one usage pattern of the video4linux kernel interface with a different, more complex usage pattern of the video4linux kernel interface (and the media controller kernel interface) + potential userspace processing. So libcamera is absolutely "on top of" video4linux is my point. [1] https://www.kernel.org/doc/html/latest/userspace-api/media/index.html https://www.kernel.org/doc/html/latest/userspace-api/media/i...
    - HarHarVeryFunny 4y ago
      OK I'll take your word for it (libcamera using v4l2), but are you sure there arn't also any cameras supported by libcamera directly? There must be some reason for libcamera's v4l2 compatability layer ... I hadn't heard of libcamera until reading this story/thread, and it seems to provide useful functionality, so I'm trying to understand the amount of apparent hostility there seems to be towards it.. saying it should be part of v4l2. It certainly seems to provide added functionality "over and above" v4l2, and makes sense if it uses v4l2 for lower level camera access, although from a user perspective that really makes no difference.
      - mort96 4y ago
        To answer the first part: There's a lot of software out there which iterates through `/dev/videoX` devices, opens one of them (maybe by presenting the user with some UI to choose between them), and then just starts using it. You can query which pixel formats are supported and set one, you can query the range of supported frame rates and set a frame rate, you can query and set resolutions, etc. by interacting only with the /dev/videoX device you chose. However, this only works with some cameras; notably, it works with USB webcameras and most laptop webcameras. But in the embedded/phone world, and with some recent laptops, this can't work. There is simply no single device which is "the" camera. The camera system is a complex pipeline of different image processing nodes. Here's how the graph looks on the hardware I usually work with: https://i.imgur.com/NSmu4Tj.png https://i.imgur.com/NSmu4Tj.png. The actual camera is the ov5645 node at the top, but that's not really useful in itself. You need to configure the media graph; in this case, I have set it up as ov5645 -> csiphy1 -> csid1 -> ispif1 -> vfe0_pix, and finally, /dev/video3 is a video4linux device (as opposed to a "subdev", which the others are) which an application can interact with using the video4linux IOCTLs. And importantly, you can't configure things like resolution, cropping, etc. on /dev/video3; you have to use the media control API to configure the ov5645 node's resolution, then you can additionally configure scaling and cropping and other forms of image processing on the vfe0_pix node. If you want to access your frames without doing any processing, you can instead link up the ispif1 node to a vfe0_rdiX node (RDI = Raw Dump Interface). My graph is relatively simple because only the vfe0_pix node does actual configurable image processing, but there's no reason the hardware couldn't be structured in a way where different nodes do different kinds of processing. This is the stuff libcamera understands, and the stuff which normal "open a /dev/videoX device and throw ioctls at it" style application doesn't understand. The goal is for libcamera to figure out what graph it has to build to do what you want, to figure out what image processing operations are available at your various nodes, maybe apply various post-processing steps such as debayering if that's not handled by the image processing hardware, etc. So the value of the compatibility layer is that an application which expects to just care about a /dev/videoX device can have its API calls intercepted, and libcamera will basically try to make it look to the application as if it's talking to a simple device where framerate/resolution/pixel format control/post processing/etc is all handled by the /dev/videoX device. In the background, libcamera will build the media graph, configure the graph nodes, do post processing, and whatever else is necessary to make the camera work. It basically makes a complex camera system look to the application as if it is a simple USB webcam. I hope some of this makes the problem that's being solved here a bit more clear. As to why people are hostile to it, I'd just chock that up to the general culture in Hacker News being of making fun of and demeaning other people's work, especially when they don't understand it.