Jack here from Alexa. Thanks for the feedback. Quality control was nontrivial, to put it succinctly, but we certainly always want to be better. I'll have to che

Jack here from Alexa. Thanks for the feedback. Quality control was nontrivial, to put it succinctly, but we certainly always want to be better. I'll have to check if the issues you've noticed were detected in the judgement scores. For the first utterance you mentioned, all three raters put a score of 1 for spelling_score, which means "There are 1-2 spelling errors." So that's good. Though we re-collected some utterances with low scores, we didn't have the budget to get perfect scores for all utterances. As such, we decided to include all utterances along with the scores from the 3 raters, such that users can perform filtering as they'd like. Some may want to keep the noise intact to help with training.