3 ms·
I'm not sure I really understand the advantage to AL in this context. Sure you get better performance earlier, but if you want the best performance you still ap
by woeirua 6y ago
I'm not sure I really understand the advantage to AL in this context. Sure you get better performance earlier, but if you want the best performance you still appear to have to train with the same amount of data. Given the training -> example identification -> annotation -> training loop is going to be much slower than just continuing to annotate data and then running all the data at once (for a variety of reasons), I think if you were to do an honest total time and total monetary cost comparison you would probably come out with AL being more expensive overall... Am I missing something here?
- andy99 6y agoI made a comment on this too, they are not presenting it in the best light because their example has a finite pool of data so using all of it will give the best performance. You can see the active learning curve is steep at first as the technique identifies the most useful data points to add to the training pool, but then flattens out once these are exhausted. But for situations where there is a bigger pool of unlabelled data, active learning can identify which subset should be labelled to produce the best model performance, as long as the unlabelled pool contains "valuable" examples, the curve should remain steep and ideally meet performance targets much faster than for just annotating data randomly. Also, there is some evidence that adding too many easy points to the training pool can reduce performance, see e.g. focal loss. So this could potentially mitigate that effect (mind you so could using focal loss, but that would require more labeling)