6 ms·

How computer vision is changing manufacturing in 2023

238 points by sickeythecat 4y ago

PicassoCTs 4y ago
? Does not mention Keyence and others. Honestly though, the suits i worked with, were all very dated and used hand constructed feature filters etc. to detect flaws. Usually, it was easier to adapt the environment (exclude external light etc.) instead of lengthy tuning sessions for the installer. Usually the industrial cameras were also designed, so that local maintainers could readjust them, which excluded complex programming and happened in simple wizards or excel like programming surfaces. There was no time planned in to "retrain" further once the line was running. And it was cheap and good enough that way. Thus the "cutting" edge tech seemed to be eternally 20 years behind the cutting edge in other sectors relying on machine vision.
- zwieback 4y ago
  We use "smart" cameras from Keyence and Cognex but the really interesting work tends to still be in PC-based, hand-coded vision systems. Usually hand-crafted C++ or C# code but increasingly using neural networks for some, usually non-quantitative (e.g. locating but not measuring), solutions.
  - kevin_thibedeau 4y ago
    I interviewed at Cognex 15 years ago. They eliminated their EE department down to one H-1B who broke down in tears as I tried to figure out why I was being interviewed by people with no knowledge about the job. They were solely interested in the ability to reverse engineer something without any documentation. It was clear they were just repackaging cheap SZ camera modules in overpriced yellow boxes. Everyone was glowing about their "legacy" product line from before they canned their engineers. Nothing but crickets when asked about the new stuff. The founders had constructed this weird personality cult around themselves. Glad I dodged that bullet.
    - zwieback 4y ago
      East or West Coast? I had a strange interview experience in Oregon in the late 90s but not as bad as yours from the sound of it.
  - vidanay 4y ago
    As a developer and maintainer of a PC based vision solution, I don't like smart cameras. :)
    - snerbles 4y ago
      As an integrator of vision solutions, smart cameras firmly occupy the space of "Nobody Ever Got Fired For Buying IBM" Though with recent developments in machine learning, the case for PC-based solutions is a lot easier now than before. Behind all the fluff and shiny marketing, the incumbents are very stagnant.
      - vidanay 4y ago
        15 years ago, I used to claim that there were more smart cameras sitting in engineers desk drawers than there were running in production. I think that was true until about 7-8 years ago.
        snerbles 4y ago
        Definitely not the case at my old job - we deployed a bit over 200 Cognex In-Sight cameras over my five-year stint there, almost all for bespoke inspection applications for customers. They gave me plenty of swag, but if I wanted to play with one of their cameras I'd have to go out on the production floor.
        vidanay 4y ago
        Sounds like you worked for an integrator, so it stands to reason that you had a high success rate. Cognex, Keyance, DVT, et al sold a lot of smart cameras in batches of 1 and 2 to non-vision experienced engineers based on the lie that they could bolt it up to a conveyor and in an afternoon of programming on their game controller they could be up and running and miraculously improving their quality by 30% by next Monday. I think the vast majority of these cameras never saw production.
        4y ago
- jeffbee 4y ago
  Doesn't even mention National Instruments. This article is clearly cheerleading for a bunch of startups, and wants us to be ignorant of the larger picture. Robots have been picking up and stacking junk on assembly lines since the 1990s at the latest.
syntaxing 4y ago
Unfortunately, until we really need to push high scale manufacturing back to the states, it’s not changing anything for US manufacturing. I worked for a company that took almost a decade just to change from devicenet to ethercat, predictive analytics took 5. Any sort of “smart” system just doesn’t have a huge momentum unless we’re producing items at China rate and need to maintain cost low.
- zwieback 4y ago
  Sort of agree but a couple counterexamples from my machine vision career: - Agriculture and food processing, which cannot be offshored as easily, requires very challenging machine vision solutions. Dirty environment, unpredictable lighting, unpredictable object appearance. - Proto and small scale high tech manufacturing, pre-offshoring or sensitive IP, requires machine vision solutions that are both sophisticated and quickly adaptable
  - narrator 4y ago
    Once robotics and computer vision gets there, there could be a lot of money in robotized regenerative agriculture.
    - Loughla 4y ago
      How is this wishlist related to anything that is being talked about in this thread? I'm very confused by what you're adding to the conversation. I also wish robots would do the menial labor that I do not enjoy and would take care of all of my basic needs. But this is an article about basic computer vision beginning to impact basic manufacturing. What you're talking about is decades in the future if ever. I'm very confused. Edit: The OP originally talked about an agricultural robot that could charge itself, do all the home chores, and fix things around the house. Now it's just one sentence.
      - ghaff 4y ago
        A general-purpose robotic handyman for consumers is many many decades away (at least). And if such a thing did exist it would have massive massive implications for the labor market--both on its own and because of the implications for all the other things that AI could do were such a robot possible. Computer vision in a very constrained environment is much much different and often isn't even suitable for many "simple" tasks that aren't constrained quite enough.
djfobbz 4y ago
Here’s Fanuc M-1iA series robot organizing pills by color back in 2018 @ https://youtube.com/shorts/bdosfVWhhlQ https://youtube.com/shorts/bdosfVWhhlQ …I can only imagine what they have now!
- snerbles 4y ago
  More of what they had then. That demo of real-time blob detection and sorting by color filtering was doable in 1998. Earlier than that, even. I've found about 90% of the work in vision applications in industrial packaging is in the product handling and scene setup - focal length, lens selection, exposure time, etc. - all things familiar to a photographer. The last 10% is almost always handled by bog simple algorithms that can be more or less cobbled together from OpenCV's examples and boilerplate, the most complicated usually being OCR. The value-add of these dedicated industrial vision systems is in integration. Fanuc's iRVision is good at sending spatial data back to the robot controller, but the interface itself is a horrid kludge that specifically requires Internet Explorer and in-person training at their own (admittedly very nice) facilities and promises of litigation if you so much as think about sharing documentation with co-workers. Recording images during trial runs with their native tooling was impossible, as their under-powered processor couldn't handle saving 640x480 images at 10fps while also running the vision application. So we resorted to recording test runs by feeding the live view OBS, and everyone thought I was some kind of wizard for even considering that. At least Cognex's In-Sight has the ability to simulate their weird spreadsheet-based vision programs without a camera. With Fanuc you need the whole $30,000+ robot+controller+camera setup and with real-time applications the only way to debug it is to run it in situ. Now my most recent industrial vision experience is from 2019, so maybe some things have changed. But these are folks that often don't even know what source control is and will run screaming for the hills at the first sign of anything that's not Excel or ladder logic, and balk at the idea of paying an experienced engineer more than $100k all the while wondering why they aren't finding any talent.
  - tomp 4y ago
    Sounds like there’s a gap in the market. I’m hugely enthusiastic hobbyist that would love to chat more about robotics, in particular how a hobbyist could get started with it (a robot arm + camera maybe?). I’d love to buy you virtual coffee, get in touch if you’re up to it!
bilsbie 4y ago
Is anyone using transformers in this field yet?
- snerbles 4y ago
  Smaller neural nets are commonly used in character recognition, but typical smart cameras or embedded robot controllers don't have anywhere near enough compute to run deep networks in real time. The Fanuc I was ranting about in this thread had something like 64MB of RAM in 2018. Maybe some systems are out there using Coral TPUs with custom TF Lite models. As for PC-based systems, I would be very surprised if deep learning models weren't being used in production somewhere. But in a factory environment you can go a very long way with primitive feature recognition and good control over the scene and lighting, and the customer just cares that whatever you're doing just works and any new method will have to be enough of an improvement to be worth the cost of development time.
  - vidanay 4y ago
    The 900lb gorilla in the deep learning room that everyone likes to ignore is that machine learning is horrible at providing corrective action data. Traditional machine vision is well adapted to providing statistical data such as "the diameter of the pizza is out of tolerance by 8mm" or "there are supposed to be 22 pepperonis on the pizza, but only 19 were found". Machine learning leans towards "it's not a good pizza" and doesn't provide a lot of additional data.
    - krisoft 4y ago
      > machine learning is horrible at providing corrective action data I don’t recognise the truth in what you are writing. > there are supposed to be 22 pepperonis on the pizza, but only 19 were found Instance segmentation is a solved problem. A properly constructed and trained neural network can tell you exactly how many pepperonies it sees and exactly where. Telling if that is the right number is a trivial problem from there. > the diameter of the pizza is out of tolerance by 8mm Here too, the neural network can recognise the edges of the pizza and then you can fit a shape to it. You can do this second step either with classical algorithms or with a machine learning one. (I would use a classical algorithm if the pizza is meant to be circular or rectangular shaped, and a machine learning algorithm if they are aiming for something weird, like an Italy shaped pizza or something.) > Machine learning leans towards "it's not a good pizza" Sounds like you have only heard of simple classifier models.
vidanay 4y ago
Computer vision has been deeply integrated in manufacturing for 20+ years. if you've brushed your teeth or drank a sports drink in the last 10 years, your toothbrush or bottle has probably gone through a vision system that I write the software for. (Not Cognex)
- codetrotter 4y ago
  FANUC?
  - vidanay 4y ago
    Nope.
- snarf21 4y ago
  Agreed, I worked on manufacturing quality assurance software that controlled vision to detect particles in vials of medicine 20 years ago. The main thing that has changed is the quality of the camera has greatly increased and the price of the camera has greatly decreased.
  - vidanay 4y ago
    Yeah, when I started, we were using RS-170 cameras connected to $30k Cognex acquisition boards (all analog). The switch over to USB and then GigE has been fantastic.
    - snarf21 4y ago
      I miss it in a way. USB cameras were just coming online and we're very good yet when I switched to a different industry. It looks like the company was acquired into an automation machinery parent company.
      - vidanay 4y ago
        God, the first generations of both USB and Ethernet cameras positively SUCKED. Flaky, buggy, and expensive. Our first foray into Ethernet cameras was from a company called Opteon. They had taken a stock Intel ethernet card and flashed custom firmware onto it to support their custom framing. If the cards and the cameras didn't match exact versions, you could end up bricking one or both of them. They had to be sent back to the vendor to be fixed. edit: Oh, hey! Opteon still exists! I'm sure their products are much better than those first generations. https://www.opteontech.com/products/cameras https://www.opteontech.com/products/cameras
skeletal88 4y ago
Useds few Basler GigE and USB3 camerasfor a robotics competition at the university, was fun, cameras were easy to use.. only later I saw how they are used in the industry.
FuriouslyAdrift 4y ago
In the corruagted box industry, we use camera systems to elimate missed glue lines on tabs, skewed folds, and misaligned print in real time as the material goes through the manufacturing machines. https://www.valcomelton.com/industrial-products/inspection-cameras-sensors/clearvision-camera-systems/ https://www.valcomelton.com/industrial-products/inspection-c...

238 points by sickeythecat 4y ago

? Does not mention Keyence and others. Honestly though, the suits i worked with, were all very dated and used hand constructed feature filters etc. to detect flaws. Usually, it was easier to adapt the environment (exclude external light etc.) instead of lengthy tuning sessions for the installer. Usually the industrial cameras were also designed, so that local maintainers could readjust them, which excluded complex programming and happened in simple wizards or excel like programming surfaces. There was no time planned in to "retrain" further once the line was running. And it was cheap and good enough that way. Thus the "cutting" edge tech seemed to be eternally 20 years behind the cutting edge in other sectors relying on machine vision.

zwieback 4y ago
We use "smart" cameras from Keyence and Cognex but the really interesting work tends to still be in PC-based, hand-coded vision systems. Usually hand-crafted C++ or C# code but increasingly using neural networks for some, usually non-quantitative (e.g. locating but not measuring), solutions.
- kevin_thibedeau 4y ago
  I interviewed at Cognex 15 years ago. They eliminated their EE department down to one H-1B who broke down in tears as I tried to figure out why I was being interviewed by people with no knowledge about the job. They were solely interested in the ability to reverse engineer something without any documentation. It was clear they were just repackaging cheap SZ camera modules in overpriced yellow boxes. Everyone was glowing about their "legacy" product line from before they canned their engineers. Nothing but crickets when asked about the new stuff. The founders had constructed this weird personality cult around themselves. Glad I dodged that bullet.
  - zwieback 4y ago
    East or West Coast? I had a strange interview experience in Oregon in the late 90s but not as bad as yours from the sound of it.
- vidanay 4y ago
  As a developer and maintainer of a PC based vision solution, I don't like smart cameras. :)
  - snerbles 4y ago
    As an integrator of vision solutions, smart cameras firmly occupy the space of "Nobody Ever Got Fired For Buying IBM" Though with recent developments in machine learning, the case for PC-based solutions is a lot easier now than before. Behind all the fluff and shiny marketing, the incumbents are very stagnant.
    - vidanay 4y ago
      15 years ago, I used to claim that there were more smart cameras sitting in engineers desk drawers than there were running in production. I think that was true until about 7-8 years ago.
      - snerbles 4y ago
        Definitely not the case at my old job - we deployed a bit over 200 Cognex In-Sight cameras over my five-year stint there, almost all for bespoke inspection applications for customers. They gave me plenty of swag, but if I wanted to play with one of their cameras I'd have to go out on the production floor.
        vidanay 4y ago
        Sounds like you worked for an integrator, so it stands to reason that you had a high success rate. Cognex, Keyance, DVT, et al sold a lot of smart cameras in batches of 1 and 2 to non-vision experienced engineers based on the lie that they could bolt it up to a conveyor and in an afternoon of programming on their game controller they could be up and running and miraculously improving their quality by 30% by next Monday. I think the vast majority of these cameras never saw production.
        4y ago
jeffbee 4y ago
Doesn't even mention National Instruments. This article is clearly cheerleading for a bunch of startups, and wants us to be ignorant of the larger picture. Robots have been picking up and stacking junk on assembly lines since the 1990s at the latest.

syntaxing 4y ago

Unfortunately, until we really need to push high scale manufacturing back to the states, it’s not changing anything for US manufacturing. I worked for a company that took almost a decade just to change from devicenet to ethercat, predictive analytics took 5. Any sort of “smart” system just doesn’t have a huge momentum unless we’re producing items at China rate and need to maintain cost low.

zwieback 4y ago
Sort of agree but a couple counterexamples from my machine vision career: - Agriculture and food processing, which cannot be offshored as easily, requires very challenging machine vision solutions. Dirty environment, unpredictable lighting, unpredictable object appearance. - Proto and small scale high tech manufacturing, pre-offshoring or sensitive IP, requires machine vision solutions that are both sophisticated and quickly adaptable
- narrator 4y ago
  Once robotics and computer vision gets there, there could be a lot of money in robotized regenerative agriculture.
  - Loughla 4y ago
    How is this wishlist related to anything that is being talked about in this thread? I'm very confused by what you're adding to the conversation. I also wish robots would do the menial labor that I do not enjoy and would take care of all of my basic needs. But this is an article about basic computer vision beginning to impact basic manufacturing. What you're talking about is decades in the future if ever. I'm very confused. Edit: The OP originally talked about an agricultural robot that could charge itself, do all the home chores, and fix things around the house. Now it's just one sentence.
    - ghaff 4y ago
      A general-purpose robotic handyman for consumers is many many decades away (at least). And if such a thing did exist it would have massive massive implications for the labor market--both on its own and because of the implications for all the other things that AI could do were such a robot possible. Computer vision in a very constrained environment is much much different and often isn't even suitable for many "simple" tasks that aren't constrained quite enough.

djfobbz 4y ago

Here’s Fanuc M-1iA series robot organizing pills by color back in 2018 @ https://youtube.com/shorts/bdosfVWhhlQ https://youtube.com/shorts/bdosfVWhhlQ …I can only imagine what they have now!

snerbles 4y ago
More of what they had then. That demo of real-time blob detection and sorting by color filtering was doable in 1998. Earlier than that, even. I've found about 90% of the work in vision applications in industrial packaging is in the product handling and scene setup - focal length, lens selection, exposure time, etc. - all things familiar to a photographer. The last 10% is almost always handled by bog simple algorithms that can be more or less cobbled together from OpenCV's examples and boilerplate, the most complicated usually being OCR. The value-add of these dedicated industrial vision systems is in integration. Fanuc's iRVision is good at sending spatial data back to the robot controller, but the interface itself is a horrid kludge that specifically requires Internet Explorer and in-person training at their own (admittedly very nice) facilities and promises of litigation if you so much as think about sharing documentation with co-workers. Recording images during trial runs with their native tooling was impossible, as their under-powered processor couldn't handle saving 640x480 images at 10fps while also running the vision application. So we resorted to recording test runs by feeding the live view OBS, and everyone thought I was some kind of wizard for even considering that. At least Cognex's In-Sight has the ability to simulate their weird spreadsheet-based vision programs without a camera. With Fanuc you need the whole $30,000+ robot+controller+camera setup and with real-time applications the only way to debug it is to run it in situ. Now my most recent industrial vision experience is from 2019, so maybe some things have changed. But these are folks that often don't even know what source control is and will run screaming for the hills at the first sign of anything that's not Excel or ladder logic, and balk at the idea of paying an experienced engineer more than $100k all the while wondering why they aren't finding any talent.
- tomp 4y ago
  Sounds like there’s a gap in the market. I’m hugely enthusiastic hobbyist that would love to chat more about robotics, in particular how a hobbyist could get started with it (a robot arm + camera maybe?). I’d love to buy you virtual coffee, get in touch if you’re up to it!

bilsbie 4y ago

Is anyone using transformers in this field yet?

snerbles 4y ago
Smaller neural nets are commonly used in character recognition, but typical smart cameras or embedded robot controllers don't have anywhere near enough compute to run deep networks in real time. The Fanuc I was ranting about in this thread had something like 64MB of RAM in 2018. Maybe some systems are out there using Coral TPUs with custom TF Lite models. As for PC-based systems, I would be very surprised if deep learning models weren't being used in production somewhere. But in a factory environment you can go a very long way with primitive feature recognition and good control over the scene and lighting, and the customer just cares that whatever you're doing just works and any new method will have to be enough of an improvement to be worth the cost of development time.
- vidanay 4y ago
  The 900lb gorilla in the deep learning room that everyone likes to ignore is that machine learning is horrible at providing corrective action data. Traditional machine vision is well adapted to providing statistical data such as "the diameter of the pizza is out of tolerance by 8mm" or "there are supposed to be 22 pepperonis on the pizza, but only 19 were found". Machine learning leans towards "it's not a good pizza" and doesn't provide a lot of additional data.
  - krisoft 4y ago
    > machine learning is horrible at providing corrective action data I don’t recognise the truth in what you are writing. > there are supposed to be 22 pepperonis on the pizza, but only 19 were found Instance segmentation is a solved problem. A properly constructed and trained neural network can tell you exactly how many pepperonies it sees and exactly where. Telling if that is the right number is a trivial problem from there. > the diameter of the pizza is out of tolerance by 8mm Here too, the neural network can recognise the edges of the pizza and then you can fit a shape to it. You can do this second step either with classical algorithms or with a machine learning one. (I would use a classical algorithm if the pizza is meant to be circular or rectangular shaped, and a machine learning algorithm if they are aiming for something weird, like an Italy shaped pizza or something.) > Machine learning leans towards "it's not a good pizza" Sounds like you have only heard of simple classifier models.

vidanay 4y ago

Computer vision has been deeply integrated in manufacturing for 20+ years. if you've brushed your teeth or drank a sports drink in the last 10 years, your toothbrush or bottle has probably gone through a vision system that I write the software for. (Not Cognex)

codetrotter 4y ago
FANUC?
- vidanay 4y ago
  Nope.
snarf21 4y ago
Agreed, I worked on manufacturing quality assurance software that controlled vision to detect particles in vials of medicine 20 years ago. The main thing that has changed is the quality of the camera has greatly increased and the price of the camera has greatly decreased.
- vidanay 4y ago
  Yeah, when I started, we were using RS-170 cameras connected to $30k Cognex acquisition boards (all analog). The switch over to USB and then GigE has been fantastic.
  - snarf21 4y ago
    I miss it in a way. USB cameras were just coming online and we're very good yet when I switched to a different industry. It looks like the company was acquired into an automation machinery parent company.
    - vidanay 4y ago
      God, the first generations of both USB and Ethernet cameras positively SUCKED. Flaky, buggy, and expensive. Our first foray into Ethernet cameras was from a company called Opteon. They had taken a stock Intel ethernet card and flashed custom firmware onto it to support their custom framing. If the cards and the cameras didn't match exact versions, you could end up bricking one or both of them. They had to be sent back to the vendor to be fixed. edit: Oh, hey! Opteon still exists! I'm sure their products are much better than those first generations. https://www.opteontech.com/products/cameras https://www.opteontech.com/products/cameras

skeletal88 4y ago

Useds few Basler GigE and USB3 camerasfor a robotics competition at the university, was fun, cameras were easy to use.. only later I saw how they are used in the industry.

FuriouslyAdrift 4y ago

In the corruagted box industry, we use camera systems to elimate missed glue lines on tabs, skewed folds, and misaligned print in real time as the material goes through the manufacturing machines. https://www.valcomelton.com/industrial-products/inspection-cameras-sensors/clearvision-camera-systems/ https://www.valcomelton.com/industrial-products/inspection-c...