6 ms·

OpenCV 4.0

252 points by conductor 8y ago

ChuckMcM 8y ago
Pretty amazing piece of code, the loss of the C API is sad since a number of embedded systems started there and are skittish about going to C++11. I used version 3 to build a simple license plate reader that I had stuck in an upstairs window for a time. Combine that with the simple IMSI catcher that was referenced here a while ago and you've got your own little neighborhood watch station with some pretty grown up capabilities.
- coder543 8y ago
  > the loss of the C API is sad From the article: "A lot of C API from OpenCV 1.x has been removed." That doesn't sound like they removed all of the C APIs, unless the only C API was the one that came with 1.x? I'm not very familiar with OpenCV, but since they homepage still talks about offering Python and Java APIs, it seems likely that they're going through a C interop layer somewhere. Did they never introduce new C APIs in the last three major versions? If there really is no C API, that's sad and kinda limiting. Everything binds to C, but almost nothing can bind to C++. > a number of embedded systems started there and are skittish about going to C++11 I'm curious to know what embedded systems are powerful enough to do real-time computer vision, but don't support a C++11 toolchain? C++11 is pretty old at this point, and between gcc and clang, almost everything with a 32-bit or better processor is probably supported, as well as probably many 8-bit and 16-bit architectures, but I'm skeptical about how much CV you can do on the kind of processors that are built as 16-bit processors.
  - ChuckMcM 8y ago
    It isn't a question of power, I use a lot of Cortex M machines (32 bit ARM) which can compile C++11 just fine, it is that there is a lot of infrastructure (like you really have to have a working heap system, startup files that know how to build the necessary infrastructure, Etc.) that is intimidating to embedded programmers. Not to mention screwing up the symbol names :-).
    - coder543 8y ago
      mbed makes it pretty easy to get going with C++ on Cortex-M, last time I tried it, but that was over a year ago. Still, I'm skeptical about how much CV you could actually do on a Cortex-M processor. A few dozen to a few hundred kilobytes of RAM is really limiting when we're talking about just loading up an image from a camera, let alone doing computer vision on that image, unless you pay for a Cortex-M7 to get 1MB of RAM or something really expensive like that... at which point, an entire Raspberry Pi 0 SBC with 512MB of RAM is actually cheaper than just your microcontroller alone. Of course, the microcontroller boots up instantly and supports incredibly useful sleep modes and a variety of other peripherals... but for raw performance per dollar, the M7s are just so costly.
      - ChuckMcM 8y ago
        > Still, I'm skeptical about how much CV you could actually do on a Cortex-M processor. You have seen OpenMV right? (https://openmv.io https://openmv.io) I've got one of their early cameras (waiting on my H7's).
        coder543 8y ago
        Ah, a Cortex-M7 with 1MB of RAM... just like I mentioned. haha, the earlier editions used an M4 with 256KB of RAM it looks like, but yeah. Do you think you could run the license plate reader you wrote earlier on one of these? What are the advantages you see for this over a Raspberry Pi 0 with a Raspberry Camera module? Or the new, powerful Raspberry Pi 3A+ with a camera module? This is certainly more expensive than either Pi option, with a lower resolution camera.
        ChuckMcM 8y ago
        I'm reasonably certain it can be run on the H7 but not sure about the H4. It is one of my test cases :-). Note that these chips also include a DDR memory controller so you can put 16MB of RAM on them pretty simply, some of ST Micro's 'Discovery' cards come with this populated, but that is not used on the H4 or H7 camera.
  - balp 8y ago
    Going out on the C++ support side, embedded have to bad C++ support, for many bad reasons. People tend to think C++ have much higher runtime needs that C, but used right it's usually lower, specially if you use C++-11 or even C++14. There compilers can make smart stuff that allows you to do much more in compile time and gains much in runtime. There is a few nice talks about this at CppCon including Michel Caisse. Imho the main reason is that embedded hardware chip vendors have two goals that kinda makes only supporting old C, e.g. pre c99 viable. 1) They get money on there own, or a partners toolchain, that isn't evolving as alternatives, keeping up with standards are hard. 2) They optimize the time for code to first boot time. The demo factor, not how the products work in CI/CD chain, the preformance of the code on the final product. How easy it is to get started in the sales meeting is more important.
  - rijoja 8y ago
    It's fairly easy to build a thin C layer around C++.
- symisc_devel 8y ago
  You may find SOD interesting for your embedded Computer Vision tasks. It is written in pure C and have an API similar to OpenCV. https://sod.pixlab.io https://sod.pixlab.io
- gog 8y ago
  Do you have any code for the license plate reader you can share?
- Keyframe 8y ago
  Did they really remove C API? I don't really use OpenCV. I understand linkage need, but that isn't all to it. Some of us actually use C, not C++.
chunsj 8y ago
Oh, it seems that OpenCV 4.0 does not support more easy to use/bind C API. Sad.
- nebgnahz 8y ago
  Yes, it is sad. OpenCV has a code-gen for Java/Python but they are hard to work with. A number of projects in other programming languages (Rust [1], Go [2]) has to use custom build wrappers. [1] https://github.com/nebgnahz/cv-rs https://github.com/nebgnahz/cv-rs [2] https://github.com/hybridgroup/gocv https://github.com/hybridgroup/gocv
- flukus 8y ago
  I hope someone maintains a 3.x fork.
Iv 8y ago
Oh! G-API seems nice! It looks like something I had to implement imperfectly at one point (and I suspect many OpenCV developer also did it). The ability to describe a pipeline and having it run automatically it really a good thing to have.
- lovelearning 8y ago
  You might also find Halide[1] interesting then. It too is a computation graph that does a good job of separating core algorithm and optimizations. OpenCV's neural networks module actually uses Halide as an optional backend. I found it to be a practical approach to run TF/Torch/Caffe/Darknet models on CPUs and OpenCL/AMD. There was an OpenCV proposal to port all algorithms to Halide API [1]. I think that's still the long term plan, and G-API is a kind of stopgap till then [2]. [1]: http://halide-lang.org/ http://halide-lang.org/ [2]: https://github.com/opencv/opencv/wiki/OE-23.-Module-GAPI https://github.com/opencv/opencv/wiki/OE-23.-Module-GAPI
  - Iv 8y ago
    Interesting, thanks!
kartickv 8y ago
As an iOS engineer, I remember reading that OpenCV is a few TIMES slower than alternatives like Core Image. I wish OpenCV made use of the GPU and vector hardware on phones. Is there an alternative to OpenCV that does?
- turowicz 8y ago
  It does use gpu, you need to compile it with a flag.
  - kartickv 8y ago
    Oh, which one? I was under the impression it uses only Nvidia GPUs (CUDA) on desktop.
    - Benjamin_Dobell 8y ago
      There are a heap of CMake flags that turn on various features e.g. CUDA: https://github.com/opencv/opencv/blob/4.0.0/CMakeLists.txt#L227 https://github.com/opencv/opencv/blob/4.0.0/CMakeLists.txt#L... but also OpenCL: https://github.com/opencv/opencv/blob/4.0.0/CMakeLists.txt#L267 https://github.com/opencv/opencv/blob/4.0.0/CMakeLists.txt#L... and Vulkan (which was only merged around a month ago): https://github.com/opencv/opencv/blob/4.0.0/CMakeLists.txt#L239 https://github.com/opencv/opencv/blob/4.0.0/CMakeLists.txt#L... Perhaps that last one works on mobile, although I wouldn't get your hopes up. I haven't really used OpenCV since 2.4.x, however assuming the same holds true, performance enhancements are on a feature by feature basis. Meaning that if you enable CUDA etc. then it's not going to work for everything. Not GPU, but a common compile time enhancement (at least it used to be) was to compile with Intel TBB (thread building blocks) as a lot of the APIs included optimisations.
      - kartickv 8y ago
        Thanks, but iOS supports neither CUDA, OpenCL, Vulkan nor TBB. It's only Metal and higher-level frameworks like Core Image for GPU, and vImage for SIMD image manipulation.
barbwire 8y ago
Anyone who can understand what OpenCV actually is from their front page is doing better than me.
- flukus 8y ago
  It sort of does in the first sentence "OpenCV (Open Source Computer Vision Library)", but they could really be a lot clearer about that. I really wouldn't give a crap about the license and implementation details until I had a fair idea of what the project could do for me.
- inamberclad 8y ago
  Gotta click to the about page. > OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.
- cjhanks 8y ago
  OpenCV is ubiquitous in some domains. It's like asking; "What's numpy?"
  - virtualized 8y ago
    What's numpy?
    - topspin 8y ago
      Portmanteau of Numeric and Python. It's widely used Python library for matrix math and functions for operating on large arrays and matrices.
- geezerjay 8y ago
  If for some reason you are unaware of the existence of OpenCV, just click "About" to get to a nice summary of what it's all about.
- 8y ago

252 points by conductor 8y ago

Pretty amazing piece of code, the loss of the C API is sad since a number of embedded systems started there and are skittish about going to C++11. I used version 3 to build a simple license plate reader that I had stuck in an upstairs window for a time. Combine that with the simple IMSI catcher that was referenced here a while ago and you've got your own little neighborhood watch station with some pretty grown up capabilities.

coder543 8y ago
> the loss of the C API is sad From the article: "A lot of C API from OpenCV 1.x has been removed." That doesn't sound like they removed all of the C APIs, unless the only C API was the one that came with 1.x? I'm not very familiar with OpenCV, but since they homepage still talks about offering Python and Java APIs, it seems likely that they're going through a C interop layer somewhere. Did they never introduce new C APIs in the last three major versions? If there really is no C API, that's sad and kinda limiting. Everything binds to C, but almost nothing can bind to C++. > a number of embedded systems started there and are skittish about going to C++11 I'm curious to know what embedded systems are powerful enough to do real-time computer vision, but don't support a C++11 toolchain? C++11 is pretty old at this point, and between gcc and clang, almost everything with a 32-bit or better processor is probably supported, as well as probably many 8-bit and 16-bit architectures, but I'm skeptical about how much CV you can do on the kind of processors that are built as 16-bit processors.
- ChuckMcM 8y ago
  It isn't a question of power, I use a lot of Cortex M machines (32 bit ARM) which can compile C++11 just fine, it is that there is a lot of infrastructure (like you really have to have a working heap system, startup files that know how to build the necessary infrastructure, Etc.) that is intimidating to embedded programmers. Not to mention screwing up the symbol names :-).
  - coder543 8y ago
    mbed makes it pretty easy to get going with C++ on Cortex-M, last time I tried it, but that was over a year ago. Still, I'm skeptical about how much CV you could actually do on a Cortex-M processor. A few dozen to a few hundred kilobytes of RAM is really limiting when we're talking about just loading up an image from a camera, let alone doing computer vision on that image, unless you pay for a Cortex-M7 to get 1MB of RAM or something really expensive like that... at which point, an entire Raspberry Pi 0 SBC with 512MB of RAM is actually cheaper than just your microcontroller alone. Of course, the microcontroller boots up instantly and supports incredibly useful sleep modes and a variety of other peripherals... but for raw performance per dollar, the M7s are just so costly.
    - ChuckMcM 8y ago
      > Still, I'm skeptical about how much CV you could actually do on a Cortex-M processor. You have seen OpenMV right? (https://openmv.io https://openmv.io) I've got one of their early cameras (waiting on my H7's).
      - coder543 8y ago
        Ah, a Cortex-M7 with 1MB of RAM... just like I mentioned. haha, the earlier editions used an M4 with 256KB of RAM it looks like, but yeah. Do you think you could run the license plate reader you wrote earlier on one of these? What are the advantages you see for this over a Raspberry Pi 0 with a Raspberry Camera module? Or the new, powerful Raspberry Pi 3A+ with a camera module? This is certainly more expensive than either Pi option, with a lower resolution camera.
        ChuckMcM 8y ago
        I'm reasonably certain it can be run on the H7 but not sure about the H4. It is one of my test cases :-). Note that these chips also include a DDR memory controller so you can put 16MB of RAM on them pretty simply, some of ST Micro's 'Discovery' cards come with this populated, but that is not used on the H4 or H7 camera.
- balp 8y ago
  Going out on the C++ support side, embedded have to bad C++ support, for many bad reasons. People tend to think C++ have much higher runtime needs that C, but used right it's usually lower, specially if you use C++-11 or even C++14. There compilers can make smart stuff that allows you to do much more in compile time and gains much in runtime. There is a few nice talks about this at CppCon including Michel Caisse. Imho the main reason is that embedded hardware chip vendors have two goals that kinda makes only supporting old C, e.g. pre c99 viable. 1) They get money on there own, or a partners toolchain, that isn't evolving as alternatives, keeping up with standards are hard. 2) They optimize the time for code to first boot time. The demo factor, not how the products work in CI/CD chain, the preformance of the code on the final product. How easy it is to get started in the sales meeting is more important.
- rijoja 8y ago
  It's fairly easy to build a thin C layer around C++.
symisc_devel 8y ago
You may find SOD interesting for your embedded Computer Vision tasks. It is written in pure C and have an API similar to OpenCV. https://sod.pixlab.io https://sod.pixlab.io
gog 8y ago
Do you have any code for the license plate reader you can share?
Keyframe 8y ago
Did they really remove C API? I don't really use OpenCV. I understand linkage need, but that isn't all to it. Some of us actually use C, not C++.

chunsj 8y ago

Oh, it seems that OpenCV 4.0 does not support more easy to use/bind C API. Sad.

nebgnahz 8y ago
Yes, it is sad. OpenCV has a code-gen for Java/Python but they are hard to work with. A number of projects in other programming languages (Rust [1], Go [2]) has to use custom build wrappers. [1] https://github.com/nebgnahz/cv-rs https://github.com/nebgnahz/cv-rs [2] https://github.com/hybridgroup/gocv https://github.com/hybridgroup/gocv
flukus 8y ago
I hope someone maintains a 3.x fork.

Iv 8y ago

Oh! G-API seems nice! It looks like something I had to implement imperfectly at one point (and I suspect many OpenCV developer also did it). The ability to describe a pipeline and having it run automatically it really a good thing to have.

lovelearning 8y ago
You might also find Halide[1] interesting then. It too is a computation graph that does a good job of separating core algorithm and optimizations. OpenCV's neural networks module actually uses Halide as an optional backend. I found it to be a practical approach to run TF/Torch/Caffe/Darknet models on CPUs and OpenCL/AMD. There was an OpenCV proposal to port all algorithms to Halide API [1]. I think that's still the long term plan, and G-API is a kind of stopgap till then [2]. [1]: http://halide-lang.org/ http://halide-lang.org/ [2]: https://github.com/opencv/opencv/wiki/OE-23.-Module-GAPI https://github.com/opencv/opencv/wiki/OE-23.-Module-GAPI
- Iv 8y ago
  Interesting, thanks!

kartickv 8y ago

As an iOS engineer, I remember reading that OpenCV is a few TIMES slower than alternatives like Core Image. I wish OpenCV made use of the GPU and vector hardware on phones. Is there an alternative to OpenCV that does?

turowicz 8y ago
It does use gpu, you need to compile it with a flag.
- kartickv 8y ago
  Oh, which one? I was under the impression it uses only Nvidia GPUs (CUDA) on desktop.
  - Benjamin_Dobell 8y ago
    There are a heap of CMake flags that turn on various features e.g. CUDA: https://github.com/opencv/opencv/blob/4.0.0/CMakeLists.txt#L227 https://github.com/opencv/opencv/blob/4.0.0/CMakeLists.txt#L... but also OpenCL: https://github.com/opencv/opencv/blob/4.0.0/CMakeLists.txt#L267 https://github.com/opencv/opencv/blob/4.0.0/CMakeLists.txt#L... and Vulkan (which was only merged around a month ago): https://github.com/opencv/opencv/blob/4.0.0/CMakeLists.txt#L239 https://github.com/opencv/opencv/blob/4.0.0/CMakeLists.txt#L... Perhaps that last one works on mobile, although I wouldn't get your hopes up. I haven't really used OpenCV since 2.4.x, however assuming the same holds true, performance enhancements are on a feature by feature basis. Meaning that if you enable CUDA etc. then it's not going to work for everything. Not GPU, but a common compile time enhancement (at least it used to be) was to compile with Intel TBB (thread building blocks) as a lot of the APIs included optimisations.
    - kartickv 8y ago
      Thanks, but iOS supports neither CUDA, OpenCL, Vulkan nor TBB. It's only Metal and higher-level frameworks like Core Image for GPU, and vImage for SIMD image manipulation.

barbwire 8y ago

Anyone who can understand what OpenCV actually is from their front page is doing better than me.

flukus 8y ago
It sort of does in the first sentence "OpenCV (Open Source Computer Vision Library)", but they could really be a lot clearer about that. I really wouldn't give a crap about the license and implementation details until I had a fair idea of what the project could do for me.
inamberclad 8y ago
Gotta click to the about page. > OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.
cjhanks 8y ago
OpenCV is ubiquitous in some domains. It's like asking; "What's numpy?"
- virtualized 8y ago
  What's numpy?
  - topspin 8y ago
    Portmanteau of Numeric and Python. It's widely used Python library for matrix math and functions for operating on large arrays and matrices.
geezerjay 8y ago
If for some reason you are unaware of the existence of OpenCV, just click "About" to get to a nice summary of what it's all about.
8y ago