4 ms·
> how a model feels for a user in the real world is something you can't really get by just testing against switchboard Fair but statistical systems are also ex
by gok 7y ago
> how a model feels for a user in the real world is something you can't really get by just testing against switchboard
Fair but statistical systems are also extremely susceptible to placebo. I would be curious what a test set shows.
- lunixbochs 7y agoI'm also using test sets to measure my models, including test sets from datasets I'm not training on. But I called it out because the real-world difference is _significant_ for this task, users with heavy accents previously couldn't say most mid-sized sentences without having to try again and "americanize" their voice for at least one of the words. With Common Voice mixed in, that wasn't the case.