3 ms·

Can you just tell us what the recognition tech behind it is?

• apu 15y ago
  Sure! There are 3 main pieces: 1. Segment the leaf from the background. Even though we require users to place the leaf on a white background, this is still a very tough problem because of lighting, shadows, viewpoint, blur, focus, etc. We've tried all published segmentation algorithms and nothing works. We wrote our own, using EM on the HSV colorspace as the basis for it (plus lots of heuristics for leaves). It works okay, but it definitely needs the most work. 2. Extract some features from the contour of the leaf. We use histograms of curvature over scale. The idea is that some leaves have the same rough shape, but differ at a fine scale (e.g., serrated vs. smooth leaves). Others are similar at a fine-scale but look quite different overall. To distinguish all of these cases, we estimate "curvature at a scale" at each point on the contour and build a histogram of curvature values. We then repeat this at coarser and coarser scales and concatenate all histograms together to get a large feature vector. 3. Recognition/matching: We use a simple nearest-neighbors classifier to match the feature vectors to our ground-truth database of labeled leaves. We return the top 20 ranked species or so.
  • 6ren 15y ago
    A whole other approach, but I wonder if segmentation might be easier using a video of a leaf moving against a stationary background. The leaf is the part that is changing. I've often wondered about identifying local trees; I've looked up site of trees, but often leaves seem very similar to several different trees. Great to have it for other countries, and an edible plant guide.
  • jdj 15y ago
    Why are not using any color information in the feature vector? Does it result in poor performance improvement?
    • apu 15y ago
      Color is very tricky to use, because you have to worry about white-balancing, illumination conditions, and of course leaves have a large variation in color over the course of their life.