Humans in the loop is great for this (kinda why mechanical turk was invented in the first place). I like echen's blog here http://blog.echen.me/2014/10/07/movin

Humans in the loop is great for this (kinda why mechanical turk was invented in the first place). I like echen's blog here http://blog.echen.me/2014/10/07/moving-beyond-ctr-better-recommendations-through-human-evaluation/ http://blog.echen.me/2014/10/07/moving-beyond-ctr-better-rec...