3 ms·

supervised learning: (after millions of operations you had just performed) out of 1000 you predicted label #136, but actually the true result for this input was

supervised learning: (after millions of operations you had just performed) out of 1000 you predicted label #136, but actually the true result for this input was label #25. reinforcement learning: (after millions of operations you had just performed) out of 1000 you predicted label #136. That's not right, but I won't tell you what you should have done. Also, it could have been right, but maybe you had screwed up something in one of your last 100 predictions and I won't tell you what it was. Good luck.

vonnik 11y ago
that makes sense. thanks. EDIT: followup question: to what extent does DeepMind's Deep-Q learning address the problem of associating remote causes with final outcomes?
paulsutter 11y ago
To move the analogies into general intelligence, start with the example of Elon Musk (a top notch example of GI). What drives Elon Musk? In interviews he's said he wants to be useful. He's achieving that goal in a big way. Take the moment he decided to build a rocket. It was after he had a difficult meeting to buy a russian missile. He quickly roughed out, from first principles, that he should be able to build one himself. The /drive/ to make that analysis was in pursuit of his goal (and required reinforcement learning). But what lifelong process filled his brain with all the complex and nuanced information needed to rapidly draw that conclusion? His curiosity, aka unsupervised learning, which led him to learn so many things over the years that culminated in that moment. For this reason, I might make the analogy that RL is the "conductor of the symphony", rather than calling it the "cherry on top" as LeCun does here.
psb217 11y ago
This is an overly restrictive view of RL. Yann's claim about the potential utility of RL, taken at face value, is clearly false. Backprop on a deterministic computation graph is equivalent to deterministic policy gradient in the same graph, where the reward is given by the value of the optimization objective.

vonnik 11y ago

that makes sense. thanks. EDIT: followup question: to what extent does DeepMind's Deep-Q learning address the problem of associating remote causes with final outcomes?

paulsutter 11y ago

To move the analogies into general intelligence, start with the example of Elon Musk (a top notch example of GI). What drives Elon Musk? In interviews he's said he wants to be useful. He's achieving that goal in a big way. Take the moment he decided to build a rocket. It was after he had a difficult meeting to buy a russian missile. He quickly roughed out, from first principles, that he should be able to build one himself. The /drive/ to make that analysis was in pursuit of his goal (and required reinforcement learning). But what lifelong process filled his brain with all the complex and nuanced information needed to rapidly draw that conclusion? His curiosity, aka unsupervised learning, which led him to learn so many things over the years that culminated in that moment. For this reason, I might make the analogy that RL is the "conductor of the symphony", rather than calling it the "cherry on top" as LeCun does here.

psb217 11y ago

This is an overly restrictive view of RL. Yann's claim about the potential utility of RL, taken at face value, is clearly false. Backprop on a deterministic computation graph is equivalent to deterministic policy gradient in the same graph, where the reward is given by the value of the optimization objective.