4 ms·

tl;dr: With a training dataset of over 7000 images, the system is incapable of performing the cucumber sorting task on live data with greater than 70% accuracy

tl;dr: With a training dataset of over 7000 images, the system is incapable of performing the cucumber sorting task on live data with greater than 70% accuracy - likely do to over fitting on the "small" training set (where it shows 95% accuracy). Also, it is currently incapable of taking into account a number of the existing variables used for sorting. If one were to leverage the power of a supercomputer trained with tens of thousands more observations - all meticulously hand categorized - it would probably get better. Truly, we are only limited by our imaginations.

tantalor 10y ago
How does it compare to human sorters?
- dangerlibrary 10y ago
  I understand the rhetorical implications of your question, but I am unclear on how you expect me to answer it.
  - throwaway2016a 10y ago
    I'm not sure it's rhetorical, I think the implication is that human sorters are also imperfect so what's there accuracy level? Is it 70% (a tie), 90% (clearly better), or 50% (the computer is better).
    - dangerlibrary 10y ago
      <semantic discussion about the use of "rhetorical" in my comment ensues. We land at some kind of an understanding of one another and we both grow a bit as human beings.>
    - mhb 10y ago
      So how does it do versus his mother? Regarding how much their results agree or disagree.
      - yomly 10y ago
        Well given his mother is by definition the training set, surely the machine can only perform as good as or worse than her...?
        coldtea 10y ago
        There's no reason to presume that. It's not like conveying knowledge in the sense of information transfer (where by definition a student can't know more than their teacher -- without external help). It's like teaching a skill. In human terms, a person can get to play better piano than their piano teacher, better poker than the person who taught them poker etc. As the "training set" she just gives the basic skills for the recognition. The algorithm could discover further patterns she can't see beyond what she knows (e.g. she knows just that greener ones are usually better, but the algorithm notices that greener and taller are usually also better), do the calculations much faster and consider many patterns while the woman does a more shallow examination, etc.
        dangerlibrary 10y ago
        In order to get the necessary data for a machine to bootstrap new criteria, wouldn't you need an independent scoring function for each sample? How are you going to get that data - are you going to auction off individual cucumbers? Alternatively, you could track each cucumber with an Independent ID to see when it is sold, relative to other cucumbers - on the assumption that people are selecting the best from any given batch? But then you'd need to control for things like the position of the cucumbers in the display. Now you're in pretty deep into someone else's business, because you're a wholesaler, and the data will still be insanely noisey.
        coldtea 10y ago
        >In order to get the necessary data for a machine to bootstrap new criteria, wouldn't you need an independent scoring function for each sample? Merely having an experienced human score images for them can be enough for the machine to self-discover new criteria based on the pictures and their scores.
  - bbctol 10y ago
    Well, I for one am non-rhetorically, legitimately curious about how well human sorters do.
    - dangerlibrary 10y ago
      This question is predicated on the assumption that there are objective, codified criteria against which human sorting can be judged. That assumption is false. > "In Japan, each farm has its own classification standard and there's no industry standard." Right now the only question is "how closely can the machine approximate human sorting."
      - vorotato 10y ago
        Given humans can't be judged it stands to reason that you can just call the machine a human, and bada boom, you just saved a lot of money.
        dangerlibrary 10y ago
        Humans could be judged against each farm's reference human. In this farm's case, there is only the one human sorter. That reference human is the basis for comparison for the machine as well. The machine sucks at it right now.
TN1ck 10y ago
I think he can easily extend the system to use the missing variables. Scratch marks, color, texture and prickles are easily enhanced by digital image processing. Maybe use a different network for each feature and use this output to feed into another network to get the correct classification-function.
- dangerlibrary 10y ago
  If you read the article, the majority of the limitations I describe are the result of his running the system on desktop hardware. It takes multiple days to train the system using 7,000 80x80 resolution images. Hence the caveat about the availability of a supercomputer.
  - llukas 10y ago
    Or he could stop training on CPU and use GPU like pretty much everybody in the field...

tantalor 10y ago

How does it compare to human sorters?

dangerlibrary 10y ago
I understand the rhetorical implications of your question, but I am unclear on how you expect me to answer it.
- throwaway2016a 10y ago
  I'm not sure it's rhetorical, I think the implication is that human sorters are also imperfect so what's there accuracy level? Is it 70% (a tie), 90% (clearly better), or 50% (the computer is better).
  - dangerlibrary 10y ago
    <semantic discussion about the use of "rhetorical" in my comment ensues. We land at some kind of an understanding of one another and we both grow a bit as human beings.>
  - mhb 10y ago
    So how does it do versus his mother? Regarding how much their results agree or disagree.
    - yomly 10y ago
      Well given his mother is by definition the training set, surely the machine can only perform as good as or worse than her...?
      - coldtea 10y ago
        There's no reason to presume that. It's not like conveying knowledge in the sense of information transfer (where by definition a student can't know more than their teacher -- without external help). It's like teaching a skill. In human terms, a person can get to play better piano than their piano teacher, better poker than the person who taught them poker etc. As the "training set" she just gives the basic skills for the recognition. The algorithm could discover further patterns she can't see beyond what she knows (e.g. she knows just that greener ones are usually better, but the algorithm notices that greener and taller are usually also better), do the calculations much faster and consider many patterns while the woman does a more shallow examination, etc.
        dangerlibrary 10y ago
        In order to get the necessary data for a machine to bootstrap new criteria, wouldn't you need an independent scoring function for each sample? How are you going to get that data - are you going to auction off individual cucumbers? Alternatively, you could track each cucumber with an Independent ID to see when it is sold, relative to other cucumbers - on the assumption that people are selecting the best from any given batch? But then you'd need to control for things like the position of the cucumbers in the display. Now you're in pretty deep into someone else's business, because you're a wholesaler, and the data will still be insanely noisey.
        coldtea 10y ago
        >In order to get the necessary data for a machine to bootstrap new criteria, wouldn't you need an independent scoring function for each sample? Merely having an experienced human score images for them can be enough for the machine to self-discover new criteria based on the pictures and their scores.
- bbctol 10y ago
  Well, I for one am non-rhetorically, legitimately curious about how well human sorters do.
  - dangerlibrary 10y ago
    This question is predicated on the assumption that there are objective, codified criteria against which human sorting can be judged. That assumption is false. > "In Japan, each farm has its own classification standard and there's no industry standard." Right now the only question is "how closely can the machine approximate human sorting."
    - vorotato 10y ago
      Given humans can't be judged it stands to reason that you can just call the machine a human, and bada boom, you just saved a lot of money.
      - dangerlibrary 10y ago
        Humans could be judged against each farm's reference human. In this farm's case, there is only the one human sorter. That reference human is the basis for comparison for the machine as well. The machine sucks at it right now.

TN1ck 10y ago

I think he can easily extend the system to use the missing variables. Scratch marks, color, texture and prickles are easily enhanced by digital image processing. Maybe use a different network for each feature and use this output to feed into another network to get the correct classification-function.

dangerlibrary 10y ago
If you read the article, the majority of the limitations I describe are the result of his running the system on desktop hardware. It takes multiple days to train the system using 7,000 80x80 resolution images. Hence the caveat about the availability of a supercomputer.
- llukas 10y ago
  Or he could stop training on CPU and use GPU like pretty much everybody in the field...