(grad student here who had a year of his life robbed by multi-armed bandits) It's been tried, if you want to jump in the rabbit hole check out this paper: http:

(grad student here who had a year of his life robbed by multi-armed bandits) It's been tried, if you want to jump in the rabbit hole check out this paper: http://www.kdd.org/kdd2016/subtopic/view/online-context-aware-recommendation-with-time-varying-multi-arm-bandit http://www.kdd.org/kdd2016/subtopic/view/online-context-awar...