3 ms·
It's good to hear that experts have strategies to handle the problem, but I think this will definitely be a surprise to most of the article's readers. I'm a rel
by SpicyLemonZest 6y ago
It's good to hear that experts have strategies to handle the problem, but I think this will definitely be a surprise to most of the article's readers. I'm a relatively AI-interested layman, and I had assumed that these kinds of datasets had more like 99% accuracy.
- Veedrac 6y agoHeh, I guess this is a difference in perspectives. ML has some really shoddy datasets, like really bad, worse than you think I mean. > The macro-averaged results show that the ratio of correct samples (“C”) ranges from 24% to 87%, with a large variance across the five audited datasets. Particularly severe problems were found in CCAligned and WikiMatrix, with 44 of the 65 languages that we audited for CCAligned containing under 50% correct sentences, and 19 of the 20 in WikiMatrix. In total, 15 of the 205 language specific samples (7.3%) contained not a single correct sentence. Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets — https://arxiv.org/abs/2103.12028 https://arxiv.org/abs/2103.12028 In the face of stuff like that, a 90+% correct dataset doesn't seem like a big deal.