3 ms·

I don't think I understand the rest of your comment, but I don't agree that "RL is well understood".

• nkozyra 10y ago
  Most of the ML approaches boil down to some well-defined algorithms that are at least 10 years and often 30 years old. Understanding it isn't really the issue, it's testing it for usage and tweaking it for better performance. What exactly is to "understand" about reinforcement learning (application, not theory)? You could say it's developing/emergent, but not that it's "not well understood." The rest of my comment is centered on what the real problem is: a lack of data. That will hurt any and every algorithm. If I have to make a decision/prediction and you give me a single or two pieces of input data, whatever process I employ is likely to return something poor.
  • gugagore 10y ago
    I'm not sure how close you are to working with RL/ML, possibly more than me! However, I don't think you it is useful to talk about the state of the field so broadly. If you say "supervised learning is well understood", I think you have a better chance of being right. The challenges that arise in taking sequential decisions, and learning from sequential observations is a lot harder, because all your data is temporally correlated. Yeah we have algorithms for RL, but actually there are still many theoretical things we cannot prove and in practice do not work well. I am not intimately familiar, but one example is value iteration with a non-linear value function approximation. I still maintain that RL is not well-understood, and I think evidence for that is that we don't see many examples of RL in our phones, cameras, search engines, etc. (which is not the case for supervised learning).