3 ms·
These use-cases are different. They mainly designed to workaround the issue where a differentiable loss cannot be designed. Thinking it as a e2e system for aut
by liuliu 6y ago
These use-cases are different. They mainly designed to workaround the issue where a differentiable loss cannot be designed.
Thinking it as a e2e system for autonomous driving. Your end-goal is miles-driven without accident. This metrics is impossible to work out a differentiable loss with.
Or more realistic case that actually drives Google / Facebook research in RL: increase in ads clicks or e2e purchase is hard to be integrated into a supervised learning system as the differentiable loss. The likelihood to click an ads (in a supervised learning system) is subtly different from having more ads clicks (in a reinforcement learning system).
- RLmagic 6y agoForgive me if this completely misses per the disclaimer in my first comment: In the driving example, can the differentiable loss be a derivative of a quality scoring algorithm, possibly another NN, that measures episodic quality at various time and distance-based intervals? For example, the RL systems loss/reward function attributes a quality score to every 1 second or 1/10th of a mile. My guess is that there are well known problems with this assumption. What are they? Thanks for the response!
- morei 6y agoThat's kinda just moving the problem: What is this quality scoring algorithm and how do we built it? If we had such a quality score, then the problem becomes differentiable and far easier to manage. But it's hard because we don't have (and don't know how to build) such a metric.