8 ms·
DeepMind program finds diamonds in Minecraft without being taught
- animanoir 2y ago[dead]
- theOGognf 2y agoThis looks like an article about the recent Nature publication. Was confused at first because DreamerV3 is a couple of years old now
- CodeCompost 2y agoI didn't know that Nature did movie promotions.
- protocolture 2y agoFinally a use case for AI
- jonathanyc 2y agoThey write: "Below, we show uncut videos of runs during which Dreamer collected diamonds." ... but the first video only shows the player character digging downwards without using any tools and eventually dying in lava. What?
- kbelder 2y agoProving that the AI can play Minecraft as well as my wife?
- danijar 2y agoIt gets diamonds at 1:48 in the top left video (might need to full screen to seek) [1]. The tools are admittedly really hard to see in the videos because of the timelapse and MP4 struggles a bit on the low resolution, but they are there :) [1]: https://danijar.com/dreamerv3/ https://danijar.com/dreamerv3/
- Animats 2y agoKey to Dreamer’s success, says Hafner, is that it builds a model of its surroundings and uses this ‘world model’ to ‘imagine’ future scenarios and guide decision-making. Can you look at the world model, like you can look at Waymo's world model? Or is it hidden inside weights? Machine learning with world models is very interesting, and the people doing it don't seem to say much about what the models look like. The Google manipulation work talks endlessly about the natural language user interface, but when they get to motion planning, they don't say much.
- lnsru 2y agoI implemented an acoustic segmentation system in FPGA recently. The whole world model was a long list of known events and states with feasible transitions. Plus novel things not observed before. Basically rather dumb state machine with machine learning part attached to acoustic sensors. Of course, both parts could be hidden behind weights. But state machine was easily readable and that was the biggest advantage of it.
- mnky9800n 2y agoWhy would an accounting system need acoustic sensors?
- lnsru 2y agoSorry. Terrible typo. Acoustic system was cheap though.
- mnky9800n 2y agoOh haha. I work on an acoustic detection project so I was quite excited about new applications. How exactly does your machine learning model work?
- lnsru 2y agoI would say, there was not much new in this. The key part of the project was the real time approach. Acquire samples, process them, find peaks, do FFTs, sum, multiply, divide. Get a float number, turn on the proper LEDs. The data was moved in C code between DMA blocks written in VHDL. Actually far away from optimized version. But it worked. IP does not belong to me and I would like to avoid technical details. The project was ended immediately when the company we worked for offered 25000€ for all IP created during the project. Very bad joke. I am still confused, because there was massive potential in this cooperation for everybody involved.
- suddenlybananas 2y agoAn important caveat from the paper >Moreover, we follow previous work in accelerating block breaking because learning to hold a button for hundreds of consecutive steps would be infeasible for stochastic policies, allowing us to focus on the essential challenges inherent in Minecraft.
- JohnKemeny 2y agoI'm not sure it's a serious caveat if the "hint" or "control" is in the manual.
- suddenlybananas 2y agoSorry, I don't quite follow what you mean?
- franktankbank 2y agoI didn't read the manual and when I was trying to help my kid play the game I couldn't figure out how to break blocks.
- thesz 2y ago"It allows AI to understand its physical environment and also to self-improve over time, without a human having to tell it exactly what to do."
- ks1723 2y agoI my view, the 'exactly' is crucial here. They do implicitly tell the model what to do by encoding it in the reward function: In Minecraft, the team used a protocol that gave Dreamer a ‘plus one’ reward every time it completed one of 12 progressive steps involved in diamond collection — including creating planks and a furnace, mining iron and forging an iron pickaxe. This is also why I think the title of the article is slightly misleading.
- 2y ago
- N-Krause 2y agohttps://archive.is/XutGu https://archive.is/XutGu
- colechristensen 2y agoWho would have thought you could get your TAS run published in Nature if you used enough hot buzzwords. (they have been using various old-school-definition "artifical intelligence" algorithms for a long time) https://tasvideos.org/ https://tasvideos.org/
- successful23 2y agoPretty impressive. Minecraft’s a complex environment, so for an AI to figure out how to find diamonds on its own shows real progress in learning through exploration — not just pattern recognition.
- charcircuit 2y agoYou can literally find diamonds by moving the mouse down and holding left click.
- mjamesaustin 2y agoNo you can't. Blocks broken without the correct tools do not yield items. In order to find diamonds the agent must first craft an iron pickaxe, which itself requires several other steps.
- charcircuit 2y agoFinding diamonds is different than acquiring diamonds.
- fastball 2y agoIf you actually read the paper, DeepMind's program ("Dreamer") is indeed acquiring diamonds.
- fxtentacle 2y agoI guess we can look forward to a bright future where we focus 100% on work and AI will play our games /s
- TeMPOraL 2y agoOnce again, we see that it's much easier to teach machines to perceive and decide well, in many cases well above human performance - while at the same time, making machines that can navigate the same physical environment humans do, and do a variety of manual tasks that mix power and precision, remains extremely challenging. The message this sends is pretty clear: machines are better at thinking, humans are better at manual work. That is the natural division of labor that plays into strengths and weaknesses of both computers and human beings. And so, I'm sorry to say this, but the near future is that in which computers play our games and do the thinking and creative work and management (and ultimately governance), because they're going to be better at this than us, leaving us to do all the physical labor, because that's one thing we will remain better at for a while. That, or we move past the existing economic structures, so that we no longer need to worry about being competitive with AI labor. /s, but only a little.
- bigbacaloa 2y ago[dead]
- danielbln 2y agoIn a world where AI takes over cognitive work it's not far fetched to imagine a future where robotics is going to hockey stick, at which case we will either end up like citizens of The Culture, or if not citizens of the Belt.
- weatherlite 2y ago> where we focus 100% on work Lol that's crazy optimistic, what work ?
- fxtentacle 2y agoPicking up dropped pencils, for example. Robots are still hilariously bad at that. Or driving your new AI overload around the country from LAN to LAN.
- Xelynega 2y agoIsn't this DeepMind achievement from 2023?
- smokel 2y agoYup, the technical details are described in this paper from early 2023. Seems like Nature takes its time. That used to be a good thing :) https://arxiv.org/abs/2301.04104 https://arxiv.org/abs/2301.04104
- reportgunner 2y agoArticle makes it seem like finding diamonds is some kind of super complicated logical puzzle. In reality the hardest part is knowing where to look for them and what tool you need to mine them without losing them once you find them. This was given to the AI by having it watch a video that explains it. If you watch a guide on how to find diamonds it's really just a matter of getting an iron pickaxe, digging to the right depth and strip mining until you find some.
- kuu 2y agoWhile I agree with your comment, this sentence: "This was given to the AI by having it watch a video that explains it." This was not as trivial as it may seem just a few months ago...
- NVHacker 2y agoAlpha Star was also trained initially from youtube videos of pros playing Starcraft. I would argue that it was pretty trivial a few years ago.
- ismailmaj 2y agoDo you know if it was actual videos or some simpler inputs like game state and user inputs? I’d be impressed if it was the former at that time.
- johnny22 2y agostarcraft provides replay files that start with the initial game state and then every action in the game. Not user inputs, but the actions bound to them.
- rcxdude 2y agoI don't think it was videos. Almost certainly it was replay files with a bunch of work to transform them into something that could be compared to the model's outputs. (Alphastar never 'sees' the game's interface, only a transformed version of information available via an API)
- _vere 2y agoSo can i and no one needed to teach me either, but you dont see nature writing articles on it...
- johnisgood 2y agoThis is too dismissive, and there are a zillion articles of human learning.
- Aachen 2y agoYou don't want anyone to be working on replicating things humans can already do? We'll just continue tilling fields to eternity..
- rationalfaith 2y ago[dead]
- lupusreal 2y agoCharacterizing finding diamonds as "mastering" Minecraft is extremely silly. Tantamount to saying "AI masters Chess: Captures a pawn." Getting diamonds is not even close to the hardest challenge in the game, but most readers of Nature probably don't have much experience playing Minecraft so the title is actually misleading, not harmless exaggeration.
- zimpenfish 2y ago> Getting diamonds is not even close to the hardest challenge in the game Mining diamonds isn't even necessary if you build, e.g., ianxofour's iron farm on day one and trade that iron[0] with a toolsmith, armourer, and weaponsmith. You can get full diamond armour, tools, and weapons pretty quickly (probably a handful of game weeks?) [0] Main faff here is getting them off their base trade level.
- lupusreal 2y agoTrue, and if the objective is to get some raw diamonds as fast as possible demonstrating mastery of the game, I'd expect a strategy like making a boat, finding a shipwreck and then a buried treasure chest. Takes just a few minutes usually. Really though, if AI wants to impress me it needs to collect an assortment of materials and build a decent looking base. Play the way humans usually play.
- danijar 2y agoI agree with you, this is just the start and Minecraft has a lot more to offer for future research!
- nottorp 2y agoIsn't "masters" when you build a working copy of Minas Tirith or something like that?
- fine_tune 2y agoAttempting to train this on a real workload I converted over the weekend after, "step" 8M~ so far and rarely scores above 5% and most are 0% but has scored 60% once 7M~ steps ago. Adding more than 1 GPU didn't improve speed but that's pretty standard as we don't have fancy interconnect. Bit annoying they didn't use tensorboard for logging, but overall seems like a pretty cool lib - will leave it a few days and see if it can learn (no other algo has so I dont have much hope).
- sbuttgereit 2y agoThere's a YouTube channel that does a lot of videos focused on LLMs in Minecraft: https://www.youtube.com/@EmergentGarden https://www.youtube.com/@EmergentGarden I very much like the comparative approach this guy takes looking at how different LLMs fare... including how they interact together. Worth a look.
- camel-cdr 2y agoHow robust is this? Isn't something like finding dimonds in minecraft something that old-school AI could already do decently?
- breakyerself 2y agoThose were trained on human play. This had to figure it out from scratch.
- ljdtt 2y agoSlightly off-topic from the article itself, but… does anyone else feel like Nature’s cookie banner just never goes away? I have vivid memories of trying to reject cookies multiple times, eventually giving up and accepting them just to get to the article only for the banner to show up again the next time I visit. I swear it’s giving me déjà vu every single visit.. Am I the only one experiencing this, or is this just how their site works?
- FrustratedMonky 2y agoMinecraft is ubiquitous now. But I remember the alpha version, and NOBODY knew how to make a pick ax. Humans were also very bad at figuring out these steps. People were de-compiling the java and posting help guides on the internet. How to break a tree, get sticks, make a wood pick. In Alpha, that was a big deal for humans also.
- ryoshu 2y agoOr you could watch Notch build it.
- ninetyninenine 2y agoIt’s still being a stochastic parrot. Now it’s just parroting the human creativity and imagination so I’m still not impressed. If all you’re going to do is parrot things like human consciousness or human ingenuity then I will never be impressed so long that it’s just parroting.
- smokel 2y agoYou must be confused. This research is about reinforcement learning, not about large language models.
- ninetyninenine 2y agoIt's parroting human reinforcement.
- danijar 2y agoIt actually has no human data as input and learns by itself in the environment, that's the point of the accomplishment! :)
- ninetyninenine 2y agoThat's what humans do right? So it's parroting us.
- YeGoblynQueenne 2y agoReinforcement learning is very good with games. >> In Minecraft, the team used a protocol that gave Dreamer a ‘plus one’ reward every time it completed one of 12 progressive steps involved in diamond collection — including creating planks and a furnace, mining iron and forging an iron pickaxe. And that is why it is never going to work in the real world: games have clear objectives with obvious rewards. The real world, not so much.
- smokel 2y ago> games have clear objectives with obvious rewards. The real world, not so much. Tell that to the people here who are trying to turn their startup ideas into money.
- zamadatix 2y agoI don't think folks go the startup path because the steps to go from idea to making money are obvious and clear.
- SpaceManNabs 2y ago> And that is why it is never going to work in the real world: games have clear objectives with obvious rewards. The real world, not so much. I encourage you to read deepmind's work with robots.
- YeGoblynQueenne 2y agoOh I have. For example I remember this project: >> Quantitatively, the QT-Opt approach succeeded in 96% of the grasp attempts across 700 trial grasps on previously unseen objects. Compared to our previous supervised-learning based grasping approach, which had a 78% success rate, our method reduced the error rate by more than a factor of five. https://research.google/blog/scalable-deep-reinforcement-learning-for-robotic-manipulation/ https://research.google/blog/scalable-deep-reinforcement-lea... That was in 2018. So what do you think, is vision-based robotic manipulation and grasping a solved problem, seven years later? Is QT-Opt now an established industry standard in training robots with RL? Or was that just another project that was announced with great fanfare and hailed as a breakthrough that would surely lead to great increase of capabilities... only to pop, fizzle and disappear in obscurity without any real-world result, a few years later? Like most of DeepMind's RL projects do?
- deleted 2y ago[deleted]
- DeborahEmeni_ 2y agoThe “holding a button” thing actually resonated. It feels like the real work here is engineering the reward structure to make exploration even remotely viable. Dreamer’s world model might be cool, but most of the heavy lifting still seems to come from how forgiving the Minecraft environment is for training. I do wonder though: if you swapped Minecraft for a cloud-based synthetic world with similar physics but messier signals, like object permanence or social reasoning, would Dreamer still hold up? Or is it just really good at the kind of clean reward hierarchies that games offer?
- textlapse 2y agoCould this perform better by having the internal representation of Minecraft instead of raw pixels? It seems rather tenuous to keep pounding on 'training via pixels' when really a game's 2D/3D output is an optical trick at best. I understand Sergey Brin/et al had a grandiose goal for DeepMind via their Atari games challenge - but why not try alternate methods - say build/tweak games to be RL-friendly? (like MuJoCo but for games) I don't see the pixel-based approach being as applicable to the practical real world as say when software divulges its direct, internal state to the agent instead of having to fake-render to a significantly larger buffer. I understand Dreamer-like work is a great research area and one that will garner lots of citations for sure.
- EMIRELADERO 2y ago> I understand Sergey Brin/et al had a grandiose goal for DeepMind via their Atari games challenge - but why not try alternate methods - say build/tweak games to be RL-friendly? Because the ultimate goal (real-world visual intelligence) would make that impossible. There's no way to compute the "essential representation" of reality, the photons are all there is.
- textlapse 2y agoThere is no animal on planet earth that functions this way. Visual cortex and plenty of other organs compress the data into useful, semantic information before feeding into a 'neural' network. Simply from an energy and transmission perspective an animal would use up all its store to process a single frame if we were to construct such an organism based on just 'feed pixels to a giant neural network'. Things like colors, memory, objects, recognition, faces etc are all part of the equation and not some giant neural network that runs from raw photons hitting cones/rods. So this isn't biomimicry or cellular automata - it's simply a fascination similar to self-driving cars being able to drive with a image -> {neural network} -> left/right/accelerate simplification.
- janalsncm 2y agoBrains may operate on a compressed representation internally, but they only have access to their senses as inputs. A model that needs to create a viable compressed representation is quite different from one which is spoon fed one via some auxiliary data stream. Also I believe the DeepMind StarCraft model used the compressed representation, but that was a while ago. So that was already kind of solved. > simply a fascination similar to self-driving cars being able to drive with a image Whether to use lidar is more of an engineering question of the cost/benefit of adding modalities. LiDAR has come down in price quite a bit so it’s less wise in retrospect.