Disclaimer: I don't doubt that most ML folk are doing their job and are doing it well -- as a matter of fact, my company regularly does projects with our local

Disclaimer: I don't doubt that most ML folk are doing their job and are doing it well -- as a matter of fact, my company regularly does projects with our local university, so I know that, first hand. With that out of the way, I said it then and I'll say it now: a paper where Microsoft describes how well Microsoft products perform on benchmarks designed by Microsoft employees isn't science, and can't become science even if you write it in latex and put it on arxiv.