3 ms·
Ah yes, "just sample under sampled portions" The problem is that they don't exist in discrete missed data sets they only exist in larger data sets that also co
by philipwhiuk 3y ago
Ah yes, "just sample under sampled portions"
The problem is that they don't exist in discrete missed data sets they only exist in larger data sets that also contain large amounts of well sampled data.
If you could separate out the existing data you could cherry-pick the under sampled data, but do that requires the very classifier you're trying to build in the first place.
This reaction is like saying "if we knew everything we'd be able to answer every question"
- naasking 3y agoHis suggestion sounds more like human curated data sets rather than automated curation like you're assuming, although this is a good distinction to make.
- CuriouslyC 3y agoThat's not really true. You don't need to actually solve the problem, you just need to create a first order solution for optimizing your data set, which can enable better models that can be used to create a solution that is second order accurate, and so on iteratively.
- brookst 3y agoIsn’t it possible for out of band systems to improve data distribution? You don’t need a fancy model trained on all languages to tell you its training set underrepresented Finnish. You just need the training set curators to say “hey there’s no Finnish in the inputs”. If the goal is high quality models, why be a purist about only using the resulting model to measure deficiencies in the training set?