3 ms·
Nothing about a zero or few-shot classifier precludes you from validating on a data set.
by rcme 4y ago
Nothing about a zero or few-shot classifier precludes you from validating on a data set.
- nathan_compton 4y agoYou can certainly hold out a validation set while you are writing your prompt, but you can't know whether the model is over fitting for your data set since you don't know what data was in the training set.
- rcme 4y agoYou'll probably find out pretty quick in production :) But I posit that most text classification tasks don't have such strict accuracy requirements. For one, no text classifier is 100% accurate. For instance, I have genuine mail in my spam folder frequently. I see spam on social networks, etc. I struggle to think of cases that aren't at least somewhat tolerant to some amount of incorrect classification.
- jprete 4y agoI think most users are not going to do this; they have no reason to even know such a thing might be done, let alone that it’s useful.