5 ms·
That's one of these moments in life where you see tech and you know it will change the world, but don't see the problem yet.
by jsemrau 6y ago
That's one of these moments in life where you see tech and you know it will change the world, but don't see the problem yet.
- vojta_letal 6y agoDoes world really work like that?
- jsemrau 6y agoWhen the first PC with Basic launched in the 80s many people wanted to develop for it. When the iPhone Appstore launched, many people started to build apps in the ecosystem. While it might be it bit too early to compare RL to those advances in technology. I personally feel there is huge potential. I might be wrong though. And I am fine with that.
- bonoboTP 6y agoRL isn't new though, the foundational results are about 25 years old.
- WanderPanda 6y agoAnd it feels a bit like it is stalling (at least in continuous control)
- cbames89 6y agoIn my opinion there's a wide open array of approaches from control that can help with this. Learning for Control is a new conference that looks at this very topic.
- dmarchand90 6y agoThat's still very analogous to the first PCs. By that point there had been decades of foundational computer work
- stevofolife 6y agoNo one said "new". You can apply what you said to PC and iPhones. Mainframes and palms existed before them.
- bitL 6y agoRL needs a supercomputer and its code is usually too fragile - making a trivial mistake anywhere (missing a constant multiplication, swapping the order of two consecutive lines of code etc.) would likely lead to your model never converging even if you got everything else right.
- Ma8ee 6y agoWhich code is not fragile in that sense? I think that is a rather strange criticism.
- Iv 6y agoYou can do RL on an raspberry pi. Depends what problem you are trying to solve but not all of them require video analysis and billions of parameters.
- cbames89 6y agoTechnical point: Value functions that are a constant multiples of each other result in the same behavior.
- bitL 6y agoMaking a constant multiplication mistake somewhere in the code doesn't imply the new value function would be a constant multiply of the optimal one.
- chasely 6y agoThe hard part of RL for the problems I've encountered in my work is that you need a simulator. Building a reliable and accurate simulator is often an immense undertaking.
- dgb23 6y agoMaybe data scientists should team up (more?) with game programmers. They have a ton of experience in building very complex simulations.
- patrick_halina 6y agoRL is a good theoretical solution for personalization: given a user state, select an action that maximizes a long term reward (eg. revenue/engagement.) It’s tricky building the implementations because unlike Go/Chess/Atari it’s hard to simulate humans. So you have to train the agents with batches of data offline (ie. using historic data from the agent’s past actions.) This is challenging because you don’t get as many chances to try different hyper parameters. It’s starting to be used more in industry though.
- jmeister 6y agoI’ve not kept up with the recent developments in this field - is Vowpal Wabbit widely used now? Any competitors? Or do people build their own in-house systems? Thanks
- flooo 6y agoVowpal Wabbit is used but many build something in-house into existing rule-based or supervised ml systems In green fields deployments, Azure personalizer may be a nice place to start looking. On the academic side, this paper provides an overview https://content.iospress.com/articles/data-science/ds200028 https://content.iospress.com/articles/data-science/ds200028
- jmeister 6y agoThank you, the review looks helpful.