4 ms·

Frankly, the "present state of cv" as you call it, is far far ahead of the 5% classification errors you are running into on a standardized textbook problem such

by dxbydt 9y ago

Frankly, the "present state of cv" as you call it, is far far ahead of the 5% classification errors you are running into on a standardized textbook problem such as this - the Legos are all blocky discrete plastic with super-saturated colors and very distinct features. You can easily drop the error by taking multiple pictures from different angles, using ImageDataGenerator during training ( since you are using Keras), especially with horiz and vert flips, and the usual set of tuning tricks - going deeper, dropout, l2 etc. No, the real takeaway is that you have built something very impressive - a tangible physical classifier that uses vgg to get the error down to 5% with very little effort. Going from there to 0.5% would be quite easy if you post the training set so a bunch of people can try different things, see what works best for this situation. Last week I trained a car to drive around a circular track with just a stupid mnist classifier, only 9 classes, 10 lines of Keras! State of cv is very far ahead these days.

jacquesm 9y ago
That 5% is based on a relatively small set of images that is expanding rapidly (4K or so per run), so I expect that error to get lower quickly. Also, these images are stitched and so are far from perfect, the colors are all over the map due to discoloration and so on. Then there are certain nasty aliasing effects where a part will look exactly like another part from the angles the camera can look at the part. This is fixable but will require (yet) another redesign of the optics. Dropout is at .5 both for conv and connected layers. All those things you mention have already been done and yet the error is as high as it is. But this does not surprise me at all, the difference is that the pre-trained models actually do very poorly on this dataset and so I'm having to train them up from scratch. That's getting there though, another week or two and I should be in the tens of thousands of images to train with and then life will get a lot easier. Data augmentation works well and has been enabled right from the get-go. I'll definitely do another write-up in a while about the software end of things, once it crystalizes to something a bit more stable, I'm still hacking on it daily. I will post the training set at some point, first it needs to become large enough. But with the machine helping now that is getting there, rapidly.
- sorenjan 9y ago
  > Also, these images are stitched and so are far from perfect Maybe you could use a push broom scanner? I don't know if there's any hardware available for consumers, otherwise that seems ideal for a conveyor belt application.
  - jacquesm 9y ago
    This is a very rough equivalent of that. The area outside the stitched part is used to keep the belt synchronized if there are not parts in view, hence that wavy line on the belt in the video.
- grogenaut 9y ago
  thoughts on tumbling/turning the parts on the belt once identified for more pictures? It wouldn't add anything to the sort speed but it'd get you way more pictures. Thinking of that machine that tumbled dice automatically. Does the addition/removal of the belt in the background come into the recognition? Otherwise in air shots from a tumble tower or belt with trip on it would get you a lot more images.
  - jacquesm 9y ago
    The belt movement is continuous, parts that do roll around on the belt are coming out crazily warped because in each image the belt has moved 3 to 5 mm and the part has moved the same but probably in a different direction than the one the belt is moving in. Those images are useless. This scheme would work if you could always get the parts out of a single frame on the camera but they're much too long for that (24 studs long is the maximum).
    - grogenaut 9y ago
      What cameras are you using? I couldn't see it very well in the writeup. Also thanks very much for this post it's very cool. You should cross post to hackaday blog.

by dxbydt 9y ago

jacquesm 9y ago

That 5% is based on a relatively small set of images that is expanding rapidly (4K or so per run), so I expect that error to get lower quickly. Also, these images are stitched and so are far from perfect, the colors are all over the map due to discoloration and so on. Then there are certain nasty aliasing effects where a part will look exactly like another part from the angles the camera can look at the part. This is fixable but will require (yet) another redesign of the optics. Dropout is at .5 both for conv and connected layers. All those things you mention have already been done and yet the error is as high as it is. But this does not surprise me at all, the difference is that the pre-trained models actually do very poorly on this dataset and so I'm having to train them up from scratch. That's getting there though, another week or two and I should be in the tens of thousands of images to train with and then life will get a lot easier. Data augmentation works well and has been enabled right from the get-go. I'll definitely do another write-up in a while about the software end of things, once it crystalizes to something a bit more stable, I'm still hacking on it daily. I will post the training set at some point, first it needs to become large enough. But with the machine helping now that is getting there, rapidly.

sorenjan 9y ago
> Also, these images are stitched and so are far from perfect Maybe you could use a push broom scanner? I don't know if there's any hardware available for consumers, otherwise that seems ideal for a conveyor belt application.
- jacquesm 9y ago
  This is a very rough equivalent of that. The area outside the stitched part is used to keep the belt synchronized if there are not parts in view, hence that wavy line on the belt in the video.
grogenaut 9y ago
thoughts on tumbling/turning the parts on the belt once identified for more pictures? It wouldn't add anything to the sort speed but it'd get you way more pictures. Thinking of that machine that tumbled dice automatically. Does the addition/removal of the belt in the background come into the recognition? Otherwise in air shots from a tumble tower or belt with trip on it would get you a lot more images.
- jacquesm 9y ago
  The belt movement is continuous, parts that do roll around on the belt are coming out crazily warped because in each image the belt has moved 3 to 5 mm and the part has moved the same but probably in a different direction than the one the belt is moving in. Those images are useless. This scheme would work if you could always get the parts out of a single frame on the camera but they're much too long for that (24 studs long is the maximum).
  - grogenaut 9y ago
    What cameras are you using? I couldn't see it very well in the writeup. Also thanks very much for this post it's very cool. You should cross post to hackaday blog.