I would love to see a breakdown of the kinds of errors these systems make. WER is an interesting broad stroke, but it doesn't necessarily tell me how useful a g

I would love to see a breakdown of the kinds of errors these systems make. WER is an interesting broad stroke, but it doesn't necessarily tell me how useful a given system will be for some given application[0] (unless, of course, it is 0). It'd be even more interesting to see comparative error analysis across the selection of these systems. A 0.06 point improvement is certainly impressive, especially this close to the end of the scale, but I'd be curious to see if it lost anything in getting there. It's one thing if this system is strictly better than it's predecessor. It's entirely another if it is now 10% better at recognizing instances of the word 'it', but has lost the ability to distinguish 'their' and 'they're'[1]. --- [0] It is like that any viability analysis would be on an by-application basis, so I don't pretend like I'm asking for an insignificant amount of work here! [1] a crude, toy, and likely inaccurate example. Not trying to belittle the work.