3 ms·
How do you validate these classifications?
by elicksaur 2y ago
How do you validate these classifications?
- jeswin 2y agoIsn't it easier and cheaper to validate than to classify (requires expensive engineers)? I mean the skill is not as expensive - many companies do this at scale.
- scarface_74 2y agoYou need a domain expert either way. I mentioned in another reply that one of my niches is implementing call centers with Amazon Connect and Amazon Lex (the NLP engine). https://news.ycombinator.com/item?id=42748189 https://news.ycombinator.com/item?id=42748189 I don’t know the domain beforehand they are working in, I do validation testing with them.
- segmondy 2y agoThe same way you validate it if you didn't use an LLM.
- bugglebeetle 2y agoThe same way you check performance for any problem like this: by creating one or more manually-labeled test datasets, randomly sampled from the target data and looking at the resulting precision, recall, f-scores etc. LLMs change pretty much nothing about evaluation for most NLP tasks.