4 ms·
Reinforcement learning is very good with games. >> In Minecraft, the team used a protocol that gave Dreamer a ‘plus one’ reward every time it completed one of
by YeGoblynQueenne 2y ago
Reinforcement learning is very good with games.
>> In Minecraft, the team used a protocol that gave Dreamer a ‘plus one’ reward every time it completed one of 12 progressive steps involved in diamond collection — including creating planks and a furnace, mining iron and forging an iron pickaxe.
And that is why it is never going to work in the real world: games have clear objectives with obvious rewards. The real world, not so much.
- smokel 2y ago> games have clear objectives with obvious rewards. The real world, not so much. Tell that to the people here who are trying to turn their startup ideas into money.
- zamadatix 2y agoI don't think folks go the startup path because the steps to go from idea to making money are obvious and clear.
- SpaceManNabs 2y ago> And that is why it is never going to work in the real world: games have clear objectives with obvious rewards. The real world, not so much. I encourage you to read deepmind's work with robots.
- YeGoblynQueenne 2y agoOh I have. For example I remember this project: >> Quantitatively, the QT-Opt approach succeeded in 96% of the grasp attempts across 700 trial grasps on previously unseen objects. Compared to our previous supervised-learning based grasping approach, which had a 78% success rate, our method reduced the error rate by more than a factor of five. https://research.google/blog/scalable-deep-reinforcement-learning-for-robotic-manipulation/ https://research.google/blog/scalable-deep-reinforcement-lea... That was in 2018. So what do you think, is vision-based robotic manipulation and grasping a solved problem, seven years later? Is QT-Opt now an established industry standard in training robots with RL? Or was that just another project that was announced with great fanfare and hailed as a breakthrough that would surely lead to great increase of capabilities... only to pop, fizzle and disappear in obscurity without any real-world result, a few years later? Like most of DeepMind's RL projects do?
- SpaceManNabs 2y agoLet's look at 2025 https://www.youtube.com/watch?v=x-exzZ-CIUw https://www.youtube.com/watch?v=x-exzZ-CIUw It looks pretty awesome. Let's see what happens.
- YeGoblynQueenne 2y agoNice robot demo. Here's another one: https://youtu.be/03p2CADwGF8?si=BXeWXqu1_3WMS4yy https://youtu.be/03p2CADwGF8?si=BXeWXqu1_3WMS4yy A robot assembling a puzzle with machine vision! And it's only from the 1970's.
- SpaceManNabs 2y agoi dont think those demos are comparable and cool for sharing your link!
- YeGoblynQueenne 2y agoAbsolutely comparable. Consider what can be done today with hardware as powerful as in the 1970's and it's obvious that the needle hasn't budged one tick. But, like you say- let's wait and see. I always do the former but I'm still waiting for the latter.
- IshKebab 2y agoPlenty of real world situations have clear objectives with obvious rewards.
- YeGoblynQueenne 2y agoExample.
- xwolfi 2y agoWork a job, receive money
- TeMPOraL 2y agoThat's a weak example it context of at least salaried jobs, especially in context of RL, as "receive money" part is usually both significantly delayed from "work a job" part, and only loosely affected by it.
- IshKebab 2y agoThe delay between action and reward is a pretty fundamental problem with RL in general. I don't think they've come up with a really good solution yet. Of course the delay is much bigger with working a job than most RL games but fundamentally it's the same problem.
- IshKebab 2y agoFold clothes -> clothes are folded. Take children to school -> they safely arrive on time. Autonomous driving -> arrive at destination without crashing. Call centre -> customers are happy.
- TeMPOraL 2y agoThose don't look like rewards, or at least don't get processed as such for many people (myself included). Or maybe there is some art to finding happiness in simple things like having folded clothes or surviving the commute?
- janalsncm 2y ago> it is never going to work in the real world DeepSeek used RL to train R1, so that is clearly not true. But ignoring that, what is your alternative? Supervised learning? Good luck finding labels if you don’t even know what the objective is.
- YeGoblynQueenne 2y agoNo, let's not ignore DeepSeek: text is not the real world any more than Minecraft is the real world. And why do I have to offer an alternative? If it's not working, it's not working, regardless of whether there's an alternative (that we know of) or not.
- danijar 2y agoFor a lot of things, VLMs are good enough already to provide rewards. Give them the recent images and a text description of the task and ask whether the task was accomplished or not. For a more general system, you can annotate videos with text descriptions of all the tasks that have been accomplished and when, then train a reward model on those to later RL against.