It's true that not being in academia that I don't hear the conversations that fill the gaps between publications. But if one's simply going from the published

It's true that not being in academia that I don't hear the conversations that fill the gaps between publications. But if one's simply going from the published output on collaborative filtering at the moment there has been some convergence on RMSE as a benchmark. That's understandable, since it's easily measurable, and as you say, there are some folks throwing $1mm at it (which really isn't much considering what it'd do for their sales).