4 ms·

Since no one seemed believed it's almost trivial, I went ahead and implemented it: https://imgur.com/t7ivg0V https://imgur.com/t7ivg0V It took about an hour o

Since no one seemed believed it's almost trivial, I went ahead and implemented it: https://imgur.com/t7ivg0V https://imgur.com/t7ivg0V It took about an hour of Googling OpenCV questions. Also, I've never actually used OpenCV before, and don't use Python regularly. I haven't tested it against the full set of images. Even if it's not perfect you could at least use this to narrow down ones that are within some threshold, then manually verify they were tagged correctly. I'll post the code shortly.

krackers 8y ago
Any update on posting the code?
- tlrobinson 8y ago
  https://github.com/tlrobinson/skittles.py https://github.com/tlrobinson/skittles.py It's definitely not perfect but could probably be improved a bit. There are a few where strawberry and grape or strawberry and lemon are swapped. I think better lighting would help.
  - acqq 8y ago
    Nice! > I think better lighting would help. That also demonstrates the difference between having the possibility to optimize input until the output matches the known values and having to process "real life" images made under different conditions. The later is certainly more interesting: e.g. I dream of having an OCR that is "always right" no matter how bad the image is.
    - rcxdude 8y ago
      Controlled lighting is extremely helpful in computer vision and key to the operation of a lot of industrial uses of it. From a system design point of view it makes a lot of sense, but obviously it's not possible in many applications (and researchers obviously want to go after the toughest problems, and would rather mostly solve a hard problem than entirely solve an easier one).
  - dotancohen 8y ago
    Thank you, forked! I'll be learning from this.
    - tlrobinson 8y ago
      FWIW I basically just cobbled together Stack Overflow answers :)
  - MentallyRetired 8y ago
    This is awesome. What would it take to determine which brand (skittles, m&ms etc) and which sub-brand (tropical / peanut, etc) and how many calories are on the table? In my head, you would determine the relative color difference of each piece to determine the brand, but I'm not 100% clear on how that kind of ai works.
    - tlrobinson 8y ago
      Thanks. I have no idea if it's the "right" way to do this. It's the first computer vision problem I've done in a long time. With better lighting you could probably distinguish between colors of different types of candy, assuming they're different enough. If you can determine the type of candy and know how many calories are in each piece it would be easy to count the calories, of course. FYI all of your new comments are being marked as [dead] for some reason.
learnstats2 8y ago
...but it's almost trivial to count them as you go along, in addition to laying them out. Have you verified the author's claim that this is the only match in the whole data set?
- tlrobinson 8y ago
  It's much quicker to dump the bag of Skittles out and snap a photo than to arrange them and count them. My program isn't perfect, but I only spent a couple hours on it. I think it could be improved, and better lightning when taking the photos would help too. However, I was curious how well it did so I compared it to the author's results. My program misclassifies approximately 80 out of 27,740 Skittles. I searched for matches and it did indeed find the same match the author found: (0, './skittles/images/334.jpg', './skittles/images/464.jpg') (1, './skittles/images/103.jpg', './skittles/images/384.jpg') (1, './skittles/images/111.jpg', './skittles/images/134.jpg') (1, './skittles/images/118.jpg', './skittles/images/353.jpg') (1, './skittles/images/139.jpg', './skittles/images/168.jpg') (1, './skittles/images/152.jpg', './skittles/images/281.jpg') (1, './skittles/images/158.jpg', './skittles/images/244.jpg') (1, './skittles/images/198.jpg', './skittles/images/255.jpg') (1, './skittles/images/198.jpg', './skittles/images/334.jpg') (1, './skittles/images/198.jpg', './skittles/images/464.jpg') (1, './skittles/images/201.jpg', './skittles/images/338.jpg') ...etc (first row is identical, subsequent rows are one different) Since there are some errors, I did get lucky. However, if it's good enough, you could also manually verify ones that are close. Anyway, personally I'd rather write code than manually count 27,740 Skittles, but I'm grateful the author did this because it's an interesting dataset to work with.
  - possiblywrong 8y ago
    Author of the article here; I suppose I should confess that I did consider automating the counting, which did seem like an interesting problem... but I rejected the idea, for what I think is good reason: the sorting was the time-consuming part. If we skip the sorting, and depend on the code to get the counting right, and we assume that we are at worst off by one-- or more realistically, off by two, to account for the questionably-identifiable chunks of paste and such-- then how often do we need to go back for manual verification? With hindsight (although up-front simulation bore this out as well), we see that we would have to go back and manually verify anywhere from 8% (off-by-one) to 25% (off-by-two); that's every fourth pack. Now consider how much more unpleasant that manual counting is, since the Skittles are haphazardly un-arranged in the image. In short, I'm unconvinced that an automated-- while similarly accurate-- accounting would be that much more efficient.

krackers 8y ago

Any update on posting the code?

tlrobinson 8y ago
https://github.com/tlrobinson/skittles.py https://github.com/tlrobinson/skittles.py It's definitely not perfect but could probably be improved a bit. There are a few where strawberry and grape or strawberry and lemon are swapped. I think better lighting would help.
- acqq 8y ago
  Nice! > I think better lighting would help. That also demonstrates the difference between having the possibility to optimize input until the output matches the known values and having to process "real life" images made under different conditions. The later is certainly more interesting: e.g. I dream of having an OCR that is "always right" no matter how bad the image is.
  - rcxdude 8y ago
    Controlled lighting is extremely helpful in computer vision and key to the operation of a lot of industrial uses of it. From a system design point of view it makes a lot of sense, but obviously it's not possible in many applications (and researchers obviously want to go after the toughest problems, and would rather mostly solve a hard problem than entirely solve an easier one).
- dotancohen 8y ago
  Thank you, forked! I'll be learning from this.
  - tlrobinson 8y ago
    FWIW I basically just cobbled together Stack Overflow answers :)
- MentallyRetired 8y ago
  This is awesome. What would it take to determine which brand (skittles, m&ms etc) and which sub-brand (tropical / peanut, etc) and how many calories are on the table? In my head, you would determine the relative color difference of each piece to determine the brand, but I'm not 100% clear on how that kind of ai works.
  - tlrobinson 8y ago
    Thanks. I have no idea if it's the "right" way to do this. It's the first computer vision problem I've done in a long time. With better lighting you could probably distinguish between colors of different types of candy, assuming they're different enough. If you can determine the type of candy and know how many calories are in each piece it would be easy to count the calories, of course. FYI all of your new comments are being marked as [dead] for some reason.

learnstats2 8y ago

...but it's almost trivial to count them as you go along, in addition to laying them out. Have you verified the author's claim that this is the only match in the whole data set?

tlrobinson 8y ago
It's much quicker to dump the bag of Skittles out and snap a photo than to arrange them and count them. My program isn't perfect, but I only spent a couple hours on it. I think it could be improved, and better lightning when taking the photos would help too. However, I was curious how well it did so I compared it to the author's results. My program misclassifies approximately 80 out of 27,740 Skittles. I searched for matches and it did indeed find the same match the author found: (0, './skittles/images/334.jpg', './skittles/images/464.jpg') (1, './skittles/images/103.jpg', './skittles/images/384.jpg') (1, './skittles/images/111.jpg', './skittles/images/134.jpg') (1, './skittles/images/118.jpg', './skittles/images/353.jpg') (1, './skittles/images/139.jpg', './skittles/images/168.jpg') (1, './skittles/images/152.jpg', './skittles/images/281.jpg') (1, './skittles/images/158.jpg', './skittles/images/244.jpg') (1, './skittles/images/198.jpg', './skittles/images/255.jpg') (1, './skittles/images/198.jpg', './skittles/images/334.jpg') (1, './skittles/images/198.jpg', './skittles/images/464.jpg') (1, './skittles/images/201.jpg', './skittles/images/338.jpg') ...etc (first row is identical, subsequent rows are one different) Since there are some errors, I did get lucky. However, if it's good enough, you could also manually verify ones that are close. Anyway, personally I'd rather write code than manually count 27,740 Skittles, but I'm grateful the author did this because it's an interesting dataset to work with.
- possiblywrong 8y ago
  Author of the article here; I suppose I should confess that I did consider automating the counting, which did seem like an interesting problem... but I rejected the idea, for what I think is good reason: the sorting was the time-consuming part. If we skip the sorting, and depend on the code to get the counting right, and we assume that we are at worst off by one-- or more realistically, off by two, to account for the questionably-identifiable chunks of paste and such-- then how often do we need to go back for manual verification? With hindsight (although up-front simulation bore this out as well), we see that we would have to go back and manually verify anywhere from 8% (off-by-one) to 25% (off-by-two); that's every fourth pack. Now consider how much more unpleasant that manual counting is, since the Skittles are haphazardly un-arranged in the image. In short, I'm unconvinced that an automated-- while similarly accurate-- accounting would be that much more efficient.