It seems to me that much of this can be explained by one of the fundamental principles of machine learning: Training and test samples must be iid (ie independe

It seems to me that much of this can be explained by one of the fundamental principles of machine learning: Training and test samples must be iid (ie independently sampled from the same distribution) to get good performance. Otherwise there is bias. This applies not only to speakers as individuals, but to all the other factors mentioned, like context, bg noise, gender, age, accent, education, mike, emotion, etc etc. Many of the issues described can be traced down to this principle. This is why, possibly, the core issue is data. Although models and *PUs matter too, the theoretical performance boundary dependends on data quality only (with a model to match).