7 ms·
Like the models before it it struggles with my use case of tracing circuit board features. It's great with a pony on the beach but really isn't made for more ro
by bahmboo 11mo ago
Like the models before it it struggles with my use case of tracing circuit board features. It's great with a pony on the beach but really isn't made for more rote industrial type applications. With proper fine-tuning it would probably work much better but I haven't tried that yet. There are good examples on line though.
- squigz 11mo agoWow that sounds like a really interesting use-case for this. Can you link to some of those examples?
- bahmboo 11mo agoI don't have anything specific to link to but you could try it yourself with line art. Try something like a mandala or a coloring book type image. The model is trying to capture something that encompasses an entity. It isn't interested in the subfeatures of the thing. Like with a mandala it wants to segment the symbol in its entirety. It will segment some subfeatures like a leaf shaped piece but it doesn't want to segment just the lines such that it is a stencil. I hope this makes sense and I'm using terms loosely. It is an amazing model but it doesn't work for my use case, that's all!
- sneilan1 11mo agoHave you found any models that work better for your use case?
- bahmboo 11mo agoTo answer your question: no but we haven't looked because Sam is sota. Trained our own model with limited success (I'm no expert). We are pursuing a classical computer vision approach. At some level segmenting a monochrome image resembles or is actually an old fashioned flood fill - very generally. This fantastic sam model is maybe not the right fit for our application. Edit: answered the question
- grumbelbart2 11mo agoThis is a "classic" machine vision task that has traditionally been solved with non-learning algorithms. (That in part enabled the large volume, zero defect productions in electronics we have today.) There are several off-the-shelf commercial MV tools for that. Deep Learning-based methods will absolutely have a place in this in the future, but today's machines are usually classic methods. Advantages are that the hardware is much cheaper and requires less electric and thermal management. This changes these days with cheaper NPUs, but with machine lifetimes measured in decades, it will take a while.
- squigz 11mo agoMy initial thought on hearing about this was it being used for learning. It would be cool to be able to talk to an LLM about how a circuit works, what the different components are, etc.
- bahmboo 11mo agoWay late response: the off the shelf stuff is very very expensive as one would expect for industrial solutions. I was tasked to build something from scratch (our own solution). It was quite the journey and was not successful. If anyone has pointers or tips in this department I would truly love to hear about them!
- visioninmyblood 11mo agoActually a combination of LLM and VLMs could work in such cases. I just tested on some circuit boards. https://chat.vlm.run/c/f0418b26-af20-4b3d-a873-ff954f5117af https://chat.vlm.run/c/f0418b26-af20-4b3d-a873-ff954f5117af
- bahmboo 11mo agoThanks for taking the time to try that out and sharing it! Our problem is with defects on the order of 50 to 100 microns on bare boards. Defects that only a trained tech with a microscope can see - even then it's very difficult.
- visioninmyblood 11mo agoSeems like a exciting problem. Hope you get tools in the future which can solve it well.
- squigz 11mo agoIt looks like there could be a lot of potential here for learning/repair/debugging/reverse engineering. Really cool application of this stuff!
- bahmboo 11mo agoGenerally this is called automated anomaly detection.
- maurits 11mo agoI would try to take DINO v3 [1] for a spin, for that specific use-case. Or, don't laugh, the Nano Banana [2] [1]: https://github.com/facebookresearch/dinov3 https://github.com/facebookresearch/dinov3 [2]: https://imgeditor.co/ https://imgeditor.co/