11 ms·
Minecraft diamond challenge leaves AI creators stumped
- CydeWeys 7y agoInterestingly, when I started playing the game, this task was essentially impossible for human players without consulting external resources, because the crafting recipes were unlisted, and you needed to know the specific recipes for various things like crafting tables, pick-axes, and shovels that would be nearly impossible to discover by randomly trying to guess recipes. I don't know if there's a starting tutorial in the game now that explains some basic crafting recipes to get started with (there better be!), but unless the AI can understand it, or you're explicitly programming in the recipes as data, good luck.
- SquareWheel 7y agoMinecraft has had a native recipe book for some time. It displays crafting recipes as you progress through the game.
- willis936 7y agoIf the challenge was to train an AI to be able to read a recipe book that would be more impressive imo. Even more impressive would be an AI that could gather the necessary resources. It would be a full game playing AI. An imitation AI to get diamonds is neat, but the fact that it is considered feasible on one GPU in one day speaks to the difficulty of the problem more than anything imo.
- SquareWheel 7y agoIt may be cheating, and sort of against the point, but those recipes are stored as json files. Makes things a lot easier.
- CrazyStat 7y agoIt would be cheating. The contest rules [1] specifically rule out using domain knowledge. [1] https://gitlab.aicrowd.com/minerl/minerl-resources/blob/master/Rules.md https://gitlab.aicrowd.com/minerl/minerl-resources/blob/mast...
- imtringued 7y agoThe humans finding diamonds in 20 minutes are cheaters by that logic.
- willis936 7y agoAllowing access to the data changes the problem. It’s an interesting problem in its own right, but seems much easier to accomplish and arguably much less valuable than limiting input to images and output to mouse/keyboard. Teaching computers to interact with the world is the trick that we want to teach.
- usepgp 7y agoThat feature has only existed for <20% of the games lifetime. It is more than likely that the person you're replying to played the game before the recipe book existed.
- SquareWheel 7y agoYes, they said as much. I'm just letting them know that the feature they asked about is now in the game.
- jvanderbot 7y agoRight. Theres tons of prior dependencies that could be used with regular old planning and scheduling. Deciding what you need right now is actually trivial with that stock knowledge, and ML might help with where or how to get it, etc. But I dont think the problem is to get Minecraft agents that can make diamond swords ... it's to use only ML to make agents, which means throwing away all that. That's fine.
- 6gvONxR4sf7o 7y agoThe challenge is to do this via imitation learning, so it gets to watch others find diamonds first.
- CrazyStat 7y agoThey had 60 million frames of video (about 277 hours, assuming 60fps) to show to the computer for it to learn how. I think a human player could pick it up in that time.
- asdfman123 7y agoMinecraft was hugely successful early on specifically because it didn't have to explain itself -- you had to do your research to figure out the game. And when you died you sometimes lost everything. I guess they needed to add tutorials to better accomodate the wave of younger people who picked up the game in the mid 10's, but I actually appreciate Notch's decisions (or series of happy accidents that culminated in a highly addictive game).
- CydeWeys 7y agoIt really did have to explain itself though. It was frustrating trying to play without spoilers because it was basically impossible. You had to go read guides saying how to make a crafting table, how to make tools, etc.
- Erlich_Bachman 7y ago> Entrants were only allowed to use a single graphics processing unit (GPU) and four days of training time. Alphastar SCII bot has been using much more resources and time than this to train, so maybe there is one of the reasons no entrant has achieved the goal yet. > A relatively small Minecraft dataset, with 60 million frames of recorded human player data, was also made available to entrants to train their systems. It will be interesting to see if these artificial and somewhat arbitrary constraints (although I get that the idea is to restrain it to resources that are somewhat realistically available to a single individual without organizational backing today) will either cripple this challenge or in the end yield some innovative results because the entrants will have to devise algorithms that use much less data and resources than what has been traditionally required to get SOTA results. Additionally it is also unclear whether the way humans learn to play this game is actually using a smaller or a much bigger dataset to learn from. Sure, a human can learn to play it in 20 minutes, but that's after 9-10 years of other pretraining of seing, understanding and operating in the 3D physical world performing various tasks, getting compressed knowledge from other people by watching and listening to them... Maybe that would be an interesting challenge - to still constrain the final model to 1 GPU for 1 day, but at least allow the model to pretrain on arbitrary similar data, if it is not sourced directly from minecraft or any clones.
- mkagenius 7y ago> single graphics processing unit (GPU) and four days of training time. Maybe there is a reason they had this restriction? I can only think of allowing the winning AI to be ready for end users whic h generally have a single GPU?
- CrazyStat 7y agoThe resources devoted to training don't really translate to the resources needed to run the trained model. A model trained on a thousand GPUS for a week might still run in real time on a single GPU once trained. The restricted training resources are just part of the challenge. They point out that a human child can learn the necessary steps in minutes by watching someone else do it, so they wanted to see if anyone could make a computer learn it with relatively limited resources.
- soup10 7y agoTitle is misleading. Traditional AI with subroutines for each subtask would have no problem with this.
- milesvp 7y agoWas my thought as well. I’m pretty sure a stupid AI that was no more than a couple state machines and a lookup table could collect diamonds. I’ve written bots for MMO’s and the hardest part for a task like this is fighting the API the bot has access to. I suspect I could even solve this challenge 10% of the time with a blind bot that just did some brownian motion. Unless you never run into trees, every other resources is pretty easy to just dig down for. The only thing that would make this hard is the thing that makes it hard for a human player which is knowing you’re on the diamond level without using the xyz debugging info.
- twright 7y ago> hard for a human player which is knowing you’re on the diamond level That’s a really good point, lava pools start at y=10 so mining around 11 is both safe and in the diamond zone. The question is if there’s enough training data to reinforce an agent so that once it finds a (underground) lava pool it should start mining at that level.
- unoti 7y ago> The only thing that would make this hard is the thing that makes it hard for a human player which is knowing you’re on the diamond level without using the xyz debugging info. Something else that makes this much harder than it is for a human: there's no sound! You can't hear the nearby lava or water when digging down. Check out the information in the observation space-- nothing about sound in here. http://minerl.io/docs/environments/index.html#minerlobtaindiamond-v0 http://minerl.io/docs/environments/index.html#minerlobtaindi... But maybe a computer could do a better job than a human at looking at the clouds before digging, estimating how high they are, then keeping track of how many levels down we've dug. I'm pretty sure the clouds are at a fixed altitude.
- johnmoberg 7y ago
- king07828 7y agoMy google-fu is failing. I see this and copies of this article, but I want to know more about the I/O available to the AI. Did the AI get full access to the Minecraft API or were they limited audio, video, keyboard, mouse like a human player?
- CrazyStat 7y agoThe documentation [1] has details. For input they get an RGB image (i.e. what a player would see on the screen, minus the UI) and other relevant information (e.g. inventory). For output they have to specify from a list of possible actions that they can take: move forward, move back, craft an item, etc. [1] http://minerl.io/docs/tutorials/index.html http://minerl.io/docs/tutorials/index.html
- marcelsalathe 7y agoDirect link to the challenge: https://www.aicrowd.com/challenges/neurips-2019-minerl-competition https://www.aicrowd.com/challenges/neurips-2019-minerl-compe...
- uj8efdkjfdshf 7y agoI take it no one's submitted Baritone [0] yet? Or would that be against the spirit of the contest? [0] https://github.com/cabaletta/baritone https://github.com/cabaletta/baritone
- CrazyStat 7y agoThat would be against the letter of the challenge, not to mention the spirit. From the rules [1]: > The submission must train a machine learning model without relying heavily on human domain knowledge. A manually specified policy may not be used as a component of this model. Likewise, the reward function may not be changed (shaped) based on manually engineered, hard-coded functions of the state. For example, though a learned hierarchical controller is permitted, meta-controllers may not choose between two policies based on a manually specified function of the state, such as whether the agent has a certain item in its inventory. Similarly, additional rewards for approaching tree-like objects are not permitted, but rewards for encountering novel states (“curiosity rewards”) are permitted. [1] https://gitlab.aicrowd.com/minerl/minerl-resources/blob/master/Rules.md https://gitlab.aicrowd.com/minerl/minerl-resources/blob/mast...
- TheRealPomax 7y agoIt's a weird thing to somehow think that an AI with only 4 days of training data should somehow outperform even a human child, which has a decade or more of imitation learning experience. A human doesn't come to minecraft "knowing nothing", like the machine learning programs in this challenge. They already know basically everything there is to know in order to perform task-oriented goal planning and only need to focus on task discovery. If anyone had pass the bar this challenge set, they'd have basically solved the biggest hurdle in general AI, and would probably have received a call from John Carmack.
- YeGoblynQueenne 7y agoThat's kind of the point. The trend in machine learning right now is to try and solve problems "from scratch" by training "end-to-end", which means without any prior knowledge. For example, DeepMind's last three or so iterations of their Alpha-X architecture for game playing did not even have any prior knowledge of the rules of the games they tackled (chess, shoggi, go and Atari games) and were trained entirely through self-play and without anything like an opening book of moves etc, like the origional AlphaGo architecture (or DeepBlue and traditional chess engines). That is very big in AI right now, to try and remove the need for the "human in the loop" and learn complex tasks from 0 background knowledge. Obviously, like you point out, this is very difficult to do and it requires lots and lots of data and compute (no, more than that. Really lots). As the article reports, this eliminates any hopes of "democratising" machine learning and makes advancing the state of the art a game that only large corporations can play with any hope of winning.
- raverbashing 7y agoBut there is a lot of "common knowledge" that's useful All the problems solved by fine-tuning image net for example, instead of training from zero Identifying objects in a 3D environment is useful across several domains and it shouldn't be limited to Minecraft
- imtringued 7y ago>The trend in machine learning right now is to try and solve problems "from scratch" by training "end-to-end", which means without any prior knowledge. That's not a very useful goal. I can trivially make a game that is impossible for such an AI to win. It would require you to enter a 1024 bit code [0] and if you are correct you win. For a speed runner the time it takes to beat the game is merely limited by their typing speed because the code is always the same but an AI with no prior knowledge can only brute force the solution. How does the speed runner win in the first place? They ask the game developer and then post the code on a wiki. [0] an euphemism for minecrafts crafting system
- simias 7y agoI'm a bit surprised that anybody is surprised by this lack of success, that seems like a hugely complicated task given the limited amount of time and resources. The amount of possible scenarios the AI needs to be able to handle (fall damage, environmental hazards, block types behaving differently, lighting, day/night cycles and of course enemies) plus the extremely open-ended gameplay where you can almost always dig in every possible direction while collecting and crafting potentially hundreds of items seems to make it a very arduous task.
- cwyers 7y agoYeah. Look at how complex something like AlphaGo is, and then compare the complexity of Minecraft to Go. They had four days and one GPU. They never had a chance.
- tus88 7y ago> Finding a diamond in Minecraft takes many steps - from cutting trees, to making tools, to exploring caves and actually finding a diamond. Sounds like a simple state machine. Don't they program objectives into their agents? Or are they relying purely on training?
- CrazyStat 7y agoPurely training. For example, the rules give as an example of something specifically disallowed telling your agent what a minecraft tree looks like--it has to learn that.
- tus88 7y agoBut humans get to RTFM. Why are denying the autonomous agents the same right? It is this kind of oppression that will lead the machines to rise up and overthrow the human race.
- asdfman123 7y agoYes, programming it as a state machine is straightforward and there's already mods that do this. I was working on something that would automate MC a while back, too, and I programmed it as a state machine that would loop through various tasks, which consist of subtasks. When you have a goal (like gathering diamonds), that's just a list of tasks -- and then each subtask consists of a list of tasks too, until you break it down to the "point at block", "walk to block", "swing hammer" level, etc. It was fun but I am pathologically bad at finishing projects that I start.
- blt 7y agoThe agent only observes the first-person view in pixels, a compass angle, and a dictionary of inventory items [1]. So any solution would need to have a significant computer vision component. [1] http://minerl.io/docs/tutorials/first_agent.html#taking-actions http://minerl.io/docs/tutorials/first_agent.html#taking-acti...
- y1tan 7y agoGoes to contest rules : https://gitlab.aicrowd.com/minerl/minerl-resources/blob/master/Rules.md https://gitlab.aicrowd.com/minerl/minerl-resources/blob/mast... Search 'source' : - must submit source code - must submit source code - must submit source code There's your problem right there. It's not that 'creators' are stumped. It's that, given how much it would be worth pitching the same closed source to an investment group or solution seeker directly, no 'creator' with a sufficiently advanced 'new' or capable system would ever submit their source code to one of these competitions. This goes for all competitions that require participants to submit their source code. It's what you you seek or your backers after-all which is valued far more than the potential prize you're doling out. Thus, your business model. The question is always, will someone who can develop such an advanced system be dumb enough to part ways with their IP for such low value? I think not. So, you can pretty much throw any conclusions made from any such competitions in the trash. This goes for even bigger ones by bigger names. You're going to get what you'd intelligently expect : a small sampling of the same ol' same ol' approach. Don't expect any novel submissions. Don't expect any surprises. So, what's the point of this? I'm speaking about the whole site 'aicrowd' btw and any other group that organizes a 'submit your code' competition ...
- faissaloo 7y agoMinecraft is a game that is played by consulting manuals and learning from others, I would be astonished if an AI could magically discover how to find diamonds.
- detaro 7y agoFairly sure a human would "magically" figure it out with the 11 days of gameplay footage the AI got ;)
- narrator 7y agoSounds like the R/L problems they had and solved with Montezuma's Revenge.
- Buttons840 7y agoY'all got any of them AIs for turned based strategy games? Seriously, I'm starting to think turn based strategy games like Civilization, etc, will be the last to get any attention. Why? We need good AIs for these games and they're not yet solved as far as I know. Furthermore, you don't have to model the human interface by limiting actions per second like with DotA or StarCraft. Yes, we solved Go, but turn based strategy games on PC are more complicated. Seems like a worthy area to research. Does anyone know of any work on games like these?
- catalogia 7y ago>Seriously, I'm starting to think turn based strategy games like Civilization, etc, will be the last to get any attention. Why? I don't think the answer is technical. A good AI for a game like Civilization would have knowledge of human social concepts like honor and vindictiveness, which are both concepts problematic to pin down. The definitions of concepts like these are fluid, they change with the times and societies in question. But that's not to say any possible setting for these concepts is equally valid when trying to emulate human behavior. Some settings will seem unrealistic, like a mustache twirling villain or a hyper-rational pointy-eared alien. Neither make for good AIs if you're shooting for human-like behavior. Humans can certainly individually craft fictional personalities that seem realistic; authors do it all the time. But from a gameplay perspective that tactic falls flat; you end up with a limited set of personalities the player becomes familiar with and the game consequently loses relay value. In the Civilization games, every player knows that Gandhi has a short temper and likes to launch nukes. Rather than crafting individual personalities, how does a game designer define a function that returns personalities with a realistic distribution? How can a game designer define such a function if the parameters aren't truly known by science and the artists who craft individual personalities are going off wishy-washy metrics like gut instinct and artistic intuition?
- Voloskaya 7y ago> A good AI for a game like Civilization would have knowledge of human social concepts like honor and vindictiveness, This is a pretty wild assumption to say the least.
- krisroadruck 7y agoIs it screwed up that after I saw the whole "20 minutes" thing I immediately fired up Minecraft for the first time in 18 months to make sure I could still do it that fast? (18 minutes and change in case you are curious)
- datameta 7y agoThat's ~11k hours of training data
- Tempest1981 7y agoFrom what I hear, diamonds are located at random, at Y levels 0-15. The terrain is usually at Y level 50-60. So digging is required. The best strategy is to dig on levels 12-15, to avoid lava pools which are on 0-11. Not sure if the AI can toggle the debug menu, which shows the Y level, or if it can see the Y level during training. Is this a job well suited to AI, vs a canned algorithm?
- Tempest1981 7y agoVisualization of ore probability by depth: https://reddit.com/r/dataisbeautiful/comments/efvgve/where_is_each_ore_found_in_a_minecraft_world_oc/ https://reddit.com/r/dataisbeautiful/comments/efvgve/where_i... Observations: https://reddit.com/r/dataisbeautiful/comments/efvgve/_/fc2p6h3/?context=1 https://reddit.com/r/dataisbeautiful/comments/efvgve/_/fc2p6...
- atorodius 7y agoOne thing I was thinking when reading this is that minecraft is visually based on the real world. A human opening it for the first time immediately knows that a tree is a tree. Seeing someone else then cutting it down allows the brain to do a lot of connections to tree-related concepts.