4 ms·

I'll be honest... that seems like a pretty tough sell to me. You could be looking at anything from a speech disorder to a learning disability to a perfectly he

I'll be honest... that seems like a pretty tough sell to me. You could be looking at anything from a speech disorder to a learning disability to a perfectly healthy normal child with idiosyncrasies and those things are usually left to professionals with graduate degrees. The liability seems high, you've got both the potential stigma associated with incorrectly flagging a child, or worse, missing an otherwise obvious issue that means treatment is delayed during critical years.

jcims 7y ago
I totally understand where you're coming from, but the reality is that there is a lot of imbalance and delay in this type of assessment and educational support already. My wife has been an English language arts teacher and instructional coach for almost 20 years. While they definitely have standard diagnostics and regular assessments to try to keep kids on track, it's essentially best effort. And when you move out of 4th/5th grade in many districts you start getting to 90:1 teacher-student ratios with 1-2 interns available to do 1:1 reading support across maybe 500 kids. And this is in a middle class school district. There's definitely some ways it can go bad, but even basic fluency monitoring without any active remediation attempts would be a good addition.
- yorwba 7y ago
  > even basic fluency monitoring without any active remediation attempts would be a good addition. How do you define fluency? I guess it involves some combination of speed and accuracy. Speed shouldn't be too hard to measure, but for accuracy you'll probably need a model that's more accurate than 7.5% WER and can handle the difference in vocal range between children and adults. Otherwise the speech disfluencies you want to detect will be drowned out by the model not correctly recognizing actually pretty clear speech.
  - jcims 7y ago
    In the US, fluency metrics in ELA vary across standards/states but are generally coarse grained and somewhat subjective. Here's an example of the standard that a student is expected to meet for fluency in the fifth grade (bottom half): http://www.corestandards.org/ELA-Literacy/RF/5/ http://www.corestandards.org/ELA-Literacy/RF/5/ You definitely would have to use a model trained from content in the target audience (probably down to the grade level as things change so dramatically from year to year) as well as probably some labeled examples of students with various reading/speech deficiencies. This of course would lead to a lot of challenges from a privacy/regulatory/etc perspective and the entire thing would be challenging from an optics perspective (AI teachers taking over, etc etc). Edit: One thing to keep in mind with accuracy is that there is no ambiguity what the word should be, the question is how closely the utterance matches the expected sounds of the word.

jcims 7y ago

I totally understand where you're coming from, but the reality is that there is a lot of imbalance and delay in this type of assessment and educational support already. My wife has been an English language arts teacher and instructional coach for almost 20 years. While they definitely have standard diagnostics and regular assessments to try to keep kids on track, it's essentially best effort. And when you move out of 4th/5th grade in many districts you start getting to 90:1 teacher-student ratios with 1-2 interns available to do 1:1 reading support across maybe 500 kids. And this is in a middle class school district. There's definitely some ways it can go bad, but even basic fluency monitoring without any active remediation attempts would be a good addition.

yorwba 7y ago
> even basic fluency monitoring without any active remediation attempts would be a good addition. How do you define fluency? I guess it involves some combination of speed and accuracy. Speed shouldn't be too hard to measure, but for accuracy you'll probably need a model that's more accurate than 7.5% WER and can handle the difference in vocal range between children and adults. Otherwise the speech disfluencies you want to detect will be drowned out by the model not correctly recognizing actually pretty clear speech.
- jcims 7y ago
  In the US, fluency metrics in ELA vary across standards/states but are generally coarse grained and somewhat subjective. Here's an example of the standard that a student is expected to meet for fluency in the fifth grade (bottom half): http://www.corestandards.org/ELA-Literacy/RF/5/ http://www.corestandards.org/ELA-Literacy/RF/5/ You definitely would have to use a model trained from content in the target audience (probably down to the grade level as things change so dramatically from year to year) as well as probably some labeled examples of students with various reading/speech deficiencies. This of course would lead to a lot of challenges from a privacy/regulatory/etc perspective and the entire thing would be challenging from an optics perspective (AI teachers taking over, etc etc). Edit: One thing to keep in mind with accuracy is that there is no ambiguity what the word should be, the question is how closely the utterance matches the expected sounds of the word.