4 ms·

Can you publish the details of the h/w-s/w interface...the only piece I grokked was the vgg classifier. How do you go from a physical Lego on the hopper to jpg

by dxbydt 9y ago

Can you publish the details of the h/w-s/w interface...the only piece I grokked was the vgg classifier. How do you go from a physical Lego on the hopper to jpg to class label to the lego in the correct physical bin ? I'd like to do this myself. I don't have 2 tons but definitely some 10k pieces. Thanks.

jacquesm 9y ago
Hopper belt to camera belt is just a speed difference that causes the parts to become nicely spread out (at least, you hope so!), the camera stitches together frames to scan the part so a part larger than a frame can be scanned. Once the end of the part is detected it gets fed into the classifier which returns part id, category and color. Depending on what sort is set up a part then gets pushed into one of 7 bins, these periodically are emptied into larger bins and bags. If necessary a lot can be pushed through the machine twice for instance to sort parts by length or to pick out sets (that last bit works in theory but in practice there are a lot of problems to overcome because of the limited number of bins to deposit into). As for the hardware, there is a nifty little camera with a macro lens that connects to the USB port (noname Asian stuff), it has a 10x magnifying lens, a pololu servo/gpio to USB card to drive the relays and a Sainsmart 16 port relay board to drive the solenoids for the air valves. The software is all in python with a generous amount of help from the people who wrote numpy, opencv, keras and theano. The error rate is between 3 and 5% depending on how fast I set the machine, there are a number of sources for the errors, obviously classification errors, also sometimes two parts are too close to each other and even if the classifier got them right the airpuff for one pushes the other of the belt as well. To minimize this effect I keep the airpuff super short, on the order of 10 ms, which is about as fast as the solenoids can open and close reliably, but it does mean that if it misses even by a bit there is nothing to be done about it and that part will land in the 'other' bin. That error rate is still too high but with every run the classification errors go down and that's the main component. One nasty little problem was that I spaced the puffers too regular in the first iteration which meant that sometimes the parts would line up just so in the order in which they came under the camera so that more than one puffer would be active at once leading to a reduction on pressure and no parts would be pushed off the belt. That was a tricky one!
- sprocket 9y ago
  What is the name of the hardware parts that are connected to the relay? I think you've called them "puffers", but searching on that didn't bring up any results. Are you able to provide a link to where I might purchase a couple?
  - jacquesm 9y ago
    They're simply 12V solenoids with an air valve attached, to that I've run a small 1/8th" tube to a piece of thin copper tubing that's open on one end. Nothing fancy or branded, the air valves are all attached to a manifold that divides the pressure from the compressor across all the valves. 12V on the valve -> air streams out of the corresponding tube.
    - sprocket 9y ago
      Ah, yes; it was the manifold that I was looking at and wondering what it was. Thanks for the explanation!
    - deleted 9y ago
      [deleted]
- aidos 9y ago
  This is one of the greatest, most HN comments of all time! I'd love to do some hardware stuff but it always seems so inapproachable for a software-only person. "more than one puffer would be active at once" — sounds like a job for prime numbers!
  - jacquesm 9y ago
    > sounds like a job for prime numbers! And that's exactly how it was solved. The puffers are now spaced prime distances apart and that took care of that, it would be harder for a longer belt because then you'd start wasting an awful lot of space next to the belt.
    - 9y ago

by dxbydt 9y ago

jacquesm 9y ago

Hopper belt to camera belt is just a speed difference that causes the parts to become nicely spread out (at least, you hope so!), the camera stitches together frames to scan the part so a part larger than a frame can be scanned. Once the end of the part is detected it gets fed into the classifier which returns part id, category and color. Depending on what sort is set up a part then gets pushed into one of 7 bins, these periodically are emptied into larger bins and bags. If necessary a lot can be pushed through the machine twice for instance to sort parts by length or to pick out sets (that last bit works in theory but in practice there are a lot of problems to overcome because of the limited number of bins to deposit into). As for the hardware, there is a nifty little camera with a macro lens that connects to the USB port (noname Asian stuff), it has a 10x magnifying lens, a pololu servo/gpio to USB card to drive the relays and a Sainsmart 16 port relay board to drive the solenoids for the air valves. The software is all in python with a generous amount of help from the people who wrote numpy, opencv, keras and theano. The error rate is between 3 and 5% depending on how fast I set the machine, there are a number of sources for the errors, obviously classification errors, also sometimes two parts are too close to each other and even if the classifier got them right the airpuff for one pushes the other of the belt as well. To minimize this effect I keep the airpuff super short, on the order of 10 ms, which is about as fast as the solenoids can open and close reliably, but it does mean that if it misses even by a bit there is nothing to be done about it and that part will land in the 'other' bin. That error rate is still too high but with every run the classification errors go down and that's the main component. One nasty little problem was that I spaced the puffers too regular in the first iteration which meant that sometimes the parts would line up just so in the order in which they came under the camera so that more than one puffer would be active at once leading to a reduction on pressure and no parts would be pushed off the belt. That was a tricky one!

sprocket 9y ago
What is the name of the hardware parts that are connected to the relay? I think you've called them "puffers", but searching on that didn't bring up any results. Are you able to provide a link to where I might purchase a couple?
- jacquesm 9y ago
  They're simply 12V solenoids with an air valve attached, to that I've run a small 1/8th" tube to a piece of thin copper tubing that's open on one end. Nothing fancy or branded, the air valves are all attached to a manifold that divides the pressure from the compressor across all the valves. 12V on the valve -> air streams out of the corresponding tube.
  - sprocket 9y ago
    Ah, yes; it was the manifold that I was looking at and wondering what it was. Thanks for the explanation!
  - deleted 9y ago
    [deleted]
aidos 9y ago
This is one of the greatest, most HN comments of all time! I'd love to do some hardware stuff but it always seems so inapproachable for a software-only person. "more than one puffer would be active at once" — sounds like a job for prime numbers!
- jacquesm 9y ago
  > sounds like a job for prime numbers! And that's exactly how it was solved. The puffers are now spaced prime distances apart and that took care of that, it would be harder for a longer belt because then you'd start wasting an awful lot of space next to the belt.
  - 9y ago