7 ms·

Is it still relevant in the days of Tensorflow, Pytorch and other high level frameworks that solve computer vision with ease?

by suyash 3y ago

marcyb5st 3y ago
Yes, big time. For instance, once you have segmented the image you should use something like openCV to extract contours of the segmentation mask.
fleischhauf 3y ago
as far as I am aware opencv has a lot of classical Computer Vision stuff that is missing in tensorflow/pytorch. In case you need speed, or can't have data annotation, or something where deep learning is not super useful (slam) it's still very relevant I'd say.
ptero 3y ago
Absolutely. Not all computer vision problems are AI related and not all AI problems are solved by pytorch. If I need a program for, say, soft real-time processing of some industrial automation video streams to run on a low power system opencv is my #1 choice. My 2c.
- nine_k 3y ago
  OpenCV and "AI" can work well together; see YOLO: https://pjreddie.com/darknet/yolo/ https://pjreddie.com/darknet/yolo/
  - jonas21 3y ago
    In that example, they're using OpenCV to display images and read from the webcam for a demo. At one point, they make it clear that OpenCV isn't used for any of the CV parts: > We didn't compile Darknet with OpenCV so it can't display the detections directly. Instead, it saves them in predictions.png. You can open it to see the detected objects.
chillfox 3y ago
Yeah, OpenCV is really easy to work with and use in quick scripts to process images. I have used it to group images of the same subject/angle, then remove the most blurry ones.
albertzeyer 3y ago
That was also my thought. I have used OpenCV 10 years ago for some small things, but today I would simply look for any existing neural network on HuggingFace and maybe train some custom classifier on top if I need sth custom. I cannot really think of any possible case where this would not work better than what you have in OpenCV. And if speed is a concern, there are also many small models, or otherwise you can quantize them yourself, or maybe train yourself a small model with knowledge distillation. You can get this as fast and as small as you want, while still probably outperforming everything that OpenCV can offer.
- girvo 3y ago
  I can’t run those HuggingFace models on low power devices (including medium power microcontrollers). I can do OpenCV on them at the edge though.
  - albertzeyer 3y ago
    You can. For example: https://pytorch.org/edge https://pytorch.org/edge (Just an example, there is a lot more, similarly also for TensorFlow.)
    - girvo 3y ago
      I’ve tried. They simply don’t run any decent useful models on an ESP32-S3 at a speed that has real use cases. Maybe one day, but not yet. Whereas today, OpenCV and derivatives are already using the LX7 vector extensions and run fast enough to be useful.
- jpace121 3y ago
  I’ve worked on plenty of problems where all you need is to find what pixels are a specific color. While I’m sure a NN could be setup for that, why make things harder than they have to be?
  - albertzeyer 3y ago
    Why harder? My point is, by using some DL framework, you would make this simpler. You don't need to have a big NN for that, or even any NN at all, and could still use some DL framework, and I would assume this is still easier and probably faster. E.g. finding what pixels are a specific color, this would be sth like: torch.all(image == torch.tensor([255,0,0]), dim=-1).nonzero() To get all pixel pairs (x,y) where the color is red (255,0,0) of the image of shape (width,height,channels).
amelius 3y ago
Only if hallucination is not a problem for your particular application.
- filterfiber 3y ago
  I really dislike the word "hallucination" It's not a "hallucination" it's a "miss-prediction". And the more-classical methods of CV are still very capable of miss-predicting. No matter what method you choose, you must test how accurate it is, NN have the benefit of being fine-tuned for specific applications as well. If you want to be extra sure then you likely want to use both.
OJFord 3y ago
There are CV applications where techniques you'd currently get consensus on calling 'AI' would be overkill (unnecessarily intensive to run) or relatively poor performing. Say you just need to do some edge detection for example. Of course the 'AI' line is blurry and moving, probably 'computer vision' (vs. 'just processing some image data', which of course it is anyway) has been too.
iamflimflam1 3y ago
Definitely. There is still a very large class of problems that are amenable to image processing pipelines and algorithms . And a very large number of problems where there is insufficient training data to simply throw deep learning at it.
matsemann 3y ago
Some of those things could be embedded nicely into OpenCV, though. Without me having to learn about all the underlying concepts, finetune a model and learn how to deploy it on various devices. OpenCV is a wrapper around lots of common image functionality, and it could be extended to cover modern deep learning approaches.
orlp 3y ago
Even taken at face value such pipelines often still rely on OpenCV for I/O, data cleaning and preprocessing.
ilirium 3y ago
Yes, it is totally relevant. Creating a product that processes images/videos/photos typically requires composing some small models, applying classical functions to work with polygons, edges, transformation matrices to rotate/shift coordinates and pixels, resizing, working with colors, working with bright/contrast curves, open/write files, classic filters, etc. PyTorch and Tensorflow don't have this or use in their examples OpenCV functions to do it.
aitrf 3y ago
[flagged]
DonHopkins 3y ago
There are lots of ways OpenCV complements Tensorflow, pytorch, and other high level frameworks. They don't need do all those things themselves, because they assume you have OpenCV around for stuff like image io, preprocessing, manipulation, transformation, color space conversions, edge detection, filtering, segmentation, morphological transformations, feature detection, video io, real time video capture, camera calibration, 3d reconstruction, optical flow, object tracking, qr code detection, gui creation, etc.
rsp1984 3y ago
Yes. It's a different kind of computer vision. Not the kind where you just throw an expensive NN at stuff and hope for the best, but one where we're actually dealing with operations at the pixel level (corner / edge features, image warping, stitching, calibration etc). No neural nets, just very fast operations on the actual image data. It's useful either as a preprocessing step that you feed into further NN processing or -- super old school -- you are modeling your problem by hand instead of letting a NN figure it out (for better or worse).
- regularfry 3y ago
  It's also got a decent NN implementation. You can have both.
ale42 3y ago
It totally is: OpenCV does pixel-level image processing, in a deterministic and generally highly optimized way. For some computer vision, this is definitely enough, and it will have a much better performance than doing it with neural networks. It might be complementary to higher-level frameworks (typically based on machine learning models), e.g. for object detection/classification.
eurekin 3y ago
Absolutely! I'm not into that space as much lately, as I have been years ago, but in a lot of newer marketing videos I can easily recognize the OpenCV signature markings. For example from tesla videos (not the typical public facing ones, but on the NVidia tegra side) or robotic startups
j-a-a-p 3y ago
We decided for OpenCV (on a project that provided vision to a production line, think 50 images per second). The speed and computational cost advantage weighed in favour of OpenCV. I don't think this has changed so much in the last couple of years.
ChrisMarshallNY 3y ago
I know a chap that uses it for Arduino-powered drones. He is not a coder. I agree that projects like OpenCV should be better-supported. There are a couple of issues, though, that always crop up, when talking about supporting open projects: 1) I don't know of any corporation, anywhere, that actually donates for no reason at all. There's always a hook. Sometimes, it's just brand-building (logo on the free swag stuff), sometimes, it's to attract future employees (for instance, donating to a project that is maintained by a certain set of students from a college curriculum, etc.), sometimes, it's to influence "hearts and minds," and sometimes, it's a pure investment. They need a self-interested reason. 2) I don't know of any corporation, anywhere, that donates, with no expectation of influence. That's one reason why lobbying is such a big deal. There's a fig leaf of "no quid pro quo," but everyone knows that the donator is expecting a return. 3) I don't know if anyone has noticed (</s>), but the corporate environment tends to be, just a bit, on the competitive side. I know many corporations may not be willing to donate to a cause that will serve their competitors, as well as themselves. I think that foundations help. They can set up a "step and repeat" page, with donors, but keep the branding (and influence) off the actual donations.
- bee_rider 3y ago
  Corporations could donate in a way that benefits them. For example, Intel contributes a lot of code to the Linux kernel. Presumably they are at least partially contributing stuff that makes it run better on their chips. But lots of people run Linux on Intel chips. And while working on stuff that benefits Intel, their engineers are probably going to do other ancillary tasks that could be helpful. So it seems like a win-win. It should be socially acceptable for open source programs to offer the deal: we’ll consider patches from your engineers, but in exchange you have to chip in enough to support one of ours (to keep working on stuff we find interesting), and a little bit more (need some engineer-hours to review your patches). It sounds quite biased in favor of the project at first, but if the project is popular, the company could get quite a bit out of it.
  - esafak 3y ago
    Is it a bad thing if corporations donate towards features that benefit them? That is a signal to the developers to allocate their energy towards useful ends.
the__alchemist 3y ago
In addition to the other replies: Check out the official or unofficial examples for TF, PyTorch, and OpenVINO for CV applications; they all use OpenCV as a dependency, eg to get get the image data from a camera or RTSP URL, process the image, draw debug info and display the results etc.
ska 3y ago
> or high level frameworks that solve computer vision with ease? That's overstating things more than a little.

Is it still relevant in the days of Tensorflow, Pytorch and other high level frameworks that solve computer vision with ease?

by suyash 3y ago

Is it still relevant in the days of Tensorflow, Pytorch and other high level frameworks that solve computer vision with ease?

marcyb5st 3y ago

Yes, big time. For instance, once you have segmented the image you should use something like openCV to extract contours of the segmentation mask.

fleischhauf 3y ago

as far as I am aware opencv has a lot of classical Computer Vision stuff that is missing in tensorflow/pytorch. In case you need speed, or can't have data annotation, or something where deep learning is not super useful (slam) it's still very relevant I'd say.

ptero 3y ago

Absolutely. Not all computer vision problems are AI related and not all AI problems are solved by pytorch. If I need a program for, say, soft real-time processing of some industrial automation video streams to run on a low power system opencv is my #1 choice. My 2c.

nine_k 3y ago
OpenCV and "AI" can work well together; see YOLO: https://pjreddie.com/darknet/yolo/ https://pjreddie.com/darknet/yolo/
- jonas21 3y ago
  In that example, they're using OpenCV to display images and read from the webcam for a demo. At one point, they make it clear that OpenCV isn't used for any of the CV parts: > We didn't compile Darknet with OpenCV so it can't display the detections directly. Instead, it saves them in predictions.png. You can open it to see the detected objects.

chillfox 3y ago

Yeah, OpenCV is really easy to work with and use in quick scripts to process images. I have used it to group images of the same subject/angle, then remove the most blurry ones.

albertzeyer 3y ago

That was also my thought. I have used OpenCV 10 years ago for some small things, but today I would simply look for any existing neural network on HuggingFace and maybe train some custom classifier on top if I need sth custom. I cannot really think of any possible case where this would not work better than what you have in OpenCV. And if speed is a concern, there are also many small models, or otherwise you can quantize them yourself, or maybe train yourself a small model with knowledge distillation. You can get this as fast and as small as you want, while still probably outperforming everything that OpenCV can offer.

girvo 3y ago
I can’t run those HuggingFace models on low power devices (including medium power microcontrollers). I can do OpenCV on them at the edge though.
- albertzeyer 3y ago
  You can. For example: https://pytorch.org/edge https://pytorch.org/edge (Just an example, there is a lot more, similarly also for TensorFlow.)
  - girvo 3y ago
    I’ve tried. They simply don’t run any decent useful models on an ESP32-S3 at a speed that has real use cases. Maybe one day, but not yet. Whereas today, OpenCV and derivatives are already using the LX7 vector extensions and run fast enough to be useful.
jpace121 3y ago
I’ve worked on plenty of problems where all you need is to find what pixels are a specific color. While I’m sure a NN could be setup for that, why make things harder than they have to be?
- albertzeyer 3y ago
  Why harder? My point is, by using some DL framework, you would make this simpler. You don't need to have a big NN for that, or even any NN at all, and could still use some DL framework, and I would assume this is still easier and probably faster. E.g. finding what pixels are a specific color, this would be sth like: torch.all(image == torch.tensor([255,0,0]), dim=-1).nonzero() To get all pixel pairs (x,y) where the color is red (255,0,0) of the image of shape (width,height,channels).

amelius 3y ago

Only if hallucination is not a problem for your particular application.

filterfiber 3y ago
I really dislike the word "hallucination" It's not a "hallucination" it's a "miss-prediction". And the more-classical methods of CV are still very capable of miss-predicting. No matter what method you choose, you must test how accurate it is, NN have the benefit of being fine-tuned for specific applications as well. If you want to be extra sure then you likely want to use both.

OJFord 3y ago

There are CV applications where techniques you'd currently get consensus on calling 'AI' would be overkill (unnecessarily intensive to run) or relatively poor performing. Say you just need to do some edge detection for example. Of course the 'AI' line is blurry and moving, probably 'computer vision' (vs. 'just processing some image data', which of course it is anyway) has been too.

iamflimflam1 3y ago

Definitely. There is still a very large class of problems that are amenable to image processing pipelines and algorithms . And a very large number of problems where there is insufficient training data to simply throw deep learning at it.

matsemann 3y ago

Some of those things could be embedded nicely into OpenCV, though. Without me having to learn about all the underlying concepts, finetune a model and learn how to deploy it on various devices. OpenCV is a wrapper around lots of common image functionality, and it could be extended to cover modern deep learning approaches.

orlp 3y ago

Even taken at face value such pipelines often still rely on OpenCV for I/O, data cleaning and preprocessing.

ilirium 3y ago

Yes, it is totally relevant. Creating a product that processes images/videos/photos typically requires composing some small models, applying classical functions to work with polygons, edges, transformation matrices to rotate/shift coordinates and pixels, resizing, working with colors, working with bright/contrast curves, open/write files, classic filters, etc. PyTorch and Tensorflow don't have this or use in their examples OpenCV functions to do it.

aitrf 3y ago

[flagged]

DonHopkins 3y ago

There are lots of ways OpenCV complements Tensorflow, pytorch, and other high level frameworks. They don't need do all those things themselves, because they assume you have OpenCV around for stuff like image io, preprocessing, manipulation, transformation, color space conversions, edge detection, filtering, segmentation, morphological transformations, feature detection, video io, real time video capture, camera calibration, 3d reconstruction, optical flow, object tracking, qr code detection, gui creation, etc.

rsp1984 3y ago

Yes. It's a different kind of computer vision. Not the kind where you just throw an expensive NN at stuff and hope for the best, but one where we're actually dealing with operations at the pixel level (corner / edge features, image warping, stitching, calibration etc). No neural nets, just very fast operations on the actual image data. It's useful either as a preprocessing step that you feed into further NN processing or -- super old school -- you are modeling your problem by hand instead of letting a NN figure it out (for better or worse).

regularfry 3y ago
It's also got a decent NN implementation. You can have both.

ale42 3y ago

It totally is: OpenCV does pixel-level image processing, in a deterministic and generally highly optimized way. For some computer vision, this is definitely enough, and it will have a much better performance than doing it with neural networks. It might be complementary to higher-level frameworks (typically based on machine learning models), e.g. for object detection/classification.

eurekin 3y ago

Absolutely! I'm not into that space as much lately, as I have been years ago, but in a lot of newer marketing videos I can easily recognize the OpenCV signature markings. For example from tesla videos (not the typical public facing ones, but on the NVidia tegra side) or robotic startups

j-a-a-p 3y ago

We decided for OpenCV (on a project that provided vision to a production line, think 50 images per second). The speed and computational cost advantage weighed in favour of OpenCV. I don't think this has changed so much in the last couple of years.

ChrisMarshallNY 3y ago

I know a chap that uses it for Arduino-powered drones. He is not a coder. I agree that projects like OpenCV should be better-supported. There are a couple of issues, though, that always crop up, when talking about supporting open projects: 1) I don't know of any corporation, anywhere, that actually donates for no reason at all. There's always a hook. Sometimes, it's just brand-building (logo on the free swag stuff), sometimes, it's to attract future employees (for instance, donating to a project that is maintained by a certain set of students from a college curriculum, etc.), sometimes, it's to influence "hearts and minds," and sometimes, it's a pure investment. They need a self-interested reason. 2) I don't know of any corporation, anywhere, that donates, with no expectation of influence. That's one reason why lobbying is such a big deal. There's a fig leaf of "no quid pro quo," but everyone knows that the donator is expecting a return. 3) I don't know if anyone has noticed (</s>), but the corporate environment tends to be, just a bit, on the competitive side. I know many corporations may not be willing to donate to a cause that will serve their competitors, as well as themselves. I think that foundations help. They can set up a "step and repeat" page, with donors, but keep the branding (and influence) off the actual donations.

bee_rider 3y ago
Corporations could donate in a way that benefits them. For example, Intel contributes a lot of code to the Linux kernel. Presumably they are at least partially contributing stuff that makes it run better on their chips. But lots of people run Linux on Intel chips. And while working on stuff that benefits Intel, their engineers are probably going to do other ancillary tasks that could be helpful. So it seems like a win-win. It should be socially acceptable for open source programs to offer the deal: we’ll consider patches from your engineers, but in exchange you have to chip in enough to support one of ours (to keep working on stuff we find interesting), and a little bit more (need some engineer-hours to review your patches). It sounds quite biased in favor of the project at first, but if the project is popular, the company could get quite a bit out of it.
- esafak 3y ago
  Is it a bad thing if corporations donate towards features that benefit them? That is a signal to the developers to allocate their energy towards useful ends.

the__alchemist 3y ago

In addition to the other replies: Check out the official or unofficial examples for TF, PyTorch, and OpenVINO for CV applications; they all use OpenCV as a dependency, eg to get get the image data from a camera or RTSP URL, process the image, draw debug info and display the results etc.

ska 3y ago

> or high level frameworks that solve computer vision with ease? That's overstating things more than a little.