3 ms·
Nice read. Can you shed some light on what you think are the most valuable methods for identifying high entropy examples for the model to learn faster? I'm fam
by rocauc 6y ago
Nice read.
Can you shed some light on what you think are the most valuable methods for identifying high entropy examples for the model to learn faster? I'm familiar with Pool-Based Sampling, Stream-Based Selective Sampling, Membership Query Synthesis[1], but less certain which techniques are most useful in NLP.
[1] https://blog.roboflow.com/what-is-active-learning/ https://blog.roboflow.com/what-is-active-learning/
- razcle 6y agoSo entropy based active learning methods are an example of pool based sampling. Even within pool based sampling there a few different techniques. Entropy selection for pool based methods looks at the output probability for each prediction of the model in the unlabelled data-set. Then it calculates the entropy of the distributions. (in classification this is a bit like looking for the most uniform predictive distributions) and prioritises those. Entropy based active learning works ok but doesnt distinguish uncertainty that comes from a lack of knowledge (epistemic uncertainty) from noise. Techniques like Bayesian Active Learning by disagreement can do better. :)