3 ms·
Hackernews likes this as a practicable application of transfer learning. Fans of machine learning want to see transfer learning as more than a cool trick. Unfo
by groceryheist 9y ago
Hackernews likes this as a practicable application of transfer learning. Fans of machine learning want to see transfer learning as more than a cool trick.
Unfortunately, there is really no good reason for someone seriously interested in accurate information, like a researcher or journalist, to use machine learning for this particular task. Labeling a couple thousand images yourself or with friends is not that big of a task. Do it over a few evenings while watching TV and drinking beer. You could have mechanical turk workers do it for you for a few hundred dollars. In either case you will get extremely reliable information. If you use multiple judges you will have a good estimate of uncertainty for every classification. There is no way transfer learning can provide this uncertainty information.
The main advantage of this technique remains the ability to quickly label very large amounts of data on the order of hundreds of thousands of rows, or thousands of columns. For smaller data, machine learning can sometimes achieve marginal improvements in predictive performance through model complexity. However prediction in smaller data regimes is mainly useful for out-of-sample prediction. The machine learning paradigm offers limited support for measuring uncertainty for out of sample predictions, which is super important if you are a researcher.
One capability of transfer learning could be to support many many applications from one model, but I have yet to see demonstrations of this in practice. The problem is that knowing how well learning has transfered requires measuring generalizabity and so cannot be done blindly.
- anigbrowl 9y agoI don't think extreme accuracy matters that much when dealing with large sample sizes. If the margin of error is under 3% (like opinion polls) then it's good enough for many purposes.
- groceryheist 9y agoThe "margin of error" reported in opinion polls is typically just a function of the sample size and sample variance. The error is assumed to be due to statistical variation, not to problems with measurement. Measurement error is a bigger problem than statistical variation because statistical variation can often be assumed to be unbiased. My comment above is not about statistical variation, it is about measurement error. Inaccuracies in models for labeling images are a form of measurement error. Having humans label a (often stratified) sample of images is required to understand the measurement error. This requirement is the main difficulty when it comes to generalizing learning from one context to another. If you use multiple independent humans to label all your images, you can understand the errors made by the humans very well.
- anigbrowl 9y agoQuite, but my point is that good enough is sufficient for many real-world purposes, as opposed to theoretical ones.
- deleted 9y ago[deleted]