13 ms·
OpenAI bots competing against Humans right now
- bufferoverflow 8y agoFirst match is over, human pros got wrecked, and very fast.
- FartyMcFarter 8y agoThey were audience members, not pros.
- scottlegrand2 8y agoThe marketing on this whole effort is strong and the training budget was insane. That said until it defeats the best of the best it's just really amazing machine learning as opposed to game changing machine learning IMO. Also keep in mind that there's a game state API here, this is not at all like learning 2600 games from learning the screens alone which would make me fear we were on the verge of the robot apocalypse personally.
- eerikkivistik 8y agoPro game starting in a few minutes.
- deleted 8y ago[deleted]
- ionwake 8y agohi,does anyone know if openAI release an up to date state of learning for their open AI dota code? Basically can I run the same sim on my laptop and watch them play? I can see some code on Github but dont know if the actually neural net data is available too. If anyone knows the answer that will be great thanks
- eerikkivistik 8y agoLast time I looked, I did not find any published trained nets on it. As far as I know, the custom build of Dota they use isn't public either. It will most likely change in the near future though, would be the next logical step for the OpenAI team as per the goal of their organization.
- minimaxir 8y agoEven if they did release the likely-very-big model, your personal computer is likely not fast enough to make 2-3 actions per second (x5) and update weights in real time without a beefy GPU.
- eerikkivistik 8y agoI think the model itself could probably run on a PC with 64gb of RAM/VRAM (Well okay, it would be an unorthodox personal computer, I guess my point is, you could probably fit it in a Single computer, not one you have at home though). Training it definitely not possible without a large cluster.
- ionwake 8y agoThank you for the reply minimaxir - are you sure a 2015 macbook pro cannot run a compiled version of the neural net so it can play itself on my computer? Just wanted to double check - can someone else verify this before I spend weeks seeing if I can get it to work?
- deleted 8y ago[deleted]
- rhaps0dy 8y agoI'm pretty sure that a macbook pro could run the 5 agents using minimal GPU acceleration (eg WebGL) A 1024 unit LSTM only takes up a few megabytes of memory, and the multiplications at runtime are O(N^2) and not O(N^2 M), because you don't have a minibatch of updates to run.
- 8y ago
- andreyk 8y agoA good thing to read to know the basics of the game and what to pay attention to: http://smerity.com/articles/2018/n_things_to_look_out_for_in_openai_benchmark.html http://smerity.com/articles/2018/n_things_to_look_out_for_in...
- eerikkivistik 8y agoI watched the game against audience members, and the bots seemed to overextend at times. Also weird courier scouting glitches (which makes sense if all couriers are currently invulnerable for the purpose of this generation of bots). Another funny thing was watching them use smokes for no apparent reason.
- kahlonel 8y agoThey overextended in the enemy safelane. Maybe killing the enemy carry is #1 on their priority list :D
- eerikkivistik 8y agoYeah, I was thinking more in the lines of the aftermath, after diving – getting stuck in the forest behind the enemy tower. But you are correct, they seemed hella determined to get that Slark kill!
- ionwake 8y agohaha yes I saw that, it seems forest navigation is a strength though, tho sometimes they dont search
- FartyMcFarter 8y agoWard usage also seemed strange. At some point the bots put a sentry ward inside the top enemy tower's range - not sure if they gained anything from it at all.
- 8y ago
- abhiminator 8y agoThis is indescribably exciting. Can't wait to see OpenAI Five tear through the human players just like what the single bot did to 'Dendi' -- a professional gamer -- during the International 7 tournament in August last year. [0] [0] https://www.youtube.com/watch?v=wiOopO9jTZw https://www.youtube.com/watch?v=wiOopO9jTZw
- TaupeRanger 8y agoIt's not that exciting and it's very easy to beat humans at games when your reaction times are far faster and you have access to more information at a single moment in time. It's like being impressed by a calculator.
- eerikkivistik 8y agoWell yes and no. Sure, micro is a large part of Dota, but planning and coordination is worth as much. So reaction time gives you an advantage in one spectrum, but I think the real benchmark will be macro scale planning and coordination. Same applies to the Starcraft AI competition, where the bots still lose due to macro planning.
- kenning 8y agodota 2 is not as extreme about this as, say, starcraft. in starcraft, most "strategies" are heavily scripted build orders that almost never deviate from a handful of openings (similar to opening moves in chess) and a large amount of winning comes from the ability to quickly give orders to your troops ("micro"). This is an example of a strategy that is not viable without superhuman reactions: https://www.youtube.com/watch?v=IKVFZ28ybQs https://www.youtube.com/watch?v=IKVFZ28ybQs if you directly attack 20 siege tanks with 100 zerglings you will only kill about two siege tanks, but an AI can kill all the siege tanks with some zerglings left over. There's some of this in dota, but there's a cap on the skill level for most playable characters that pros generally get "close enough" to, and beyond that the strategic depth comes from area control decisionmaking. Theres over 100 heroes and many of them have really weird abilities, like the possibility of creating a temporary wall (earthshaker) or the ability to teleport anywhere on the map every 20 seconds (furion). I could be wrong though, maybe the AI is winning games by playing heroes with long range and perfectly microing them to harass and prevent the other team from ever getting gold/xp.
- wuschel 8y agoOther thread: https://news.ycombinator.com/item?id=17692577 https://news.ycombinator.com/item?id=17692577
- dantheman 8y agoCongrats OpenAI on game #1
- std_throwaway 8y agoAnd game #2!
- kmnc 8y agoVery impressed with the bots. The main strategy I see from the bots that is different from pro meta is how they spam abilities aggressively. Now the main limitation is 5 unkillable couriers which enable this. Watching, I do feel like given more practice humans would beat this version of the ai easily. It looks exploitable, and still having many rules limitations it has a ways to go. It will win in the end though, that much has become obvious.
- hackandtrip 8y agoIt would be really interesting to see what bots will do once humans will try to exploit this strategy, like adapting to the lane movement they always do after winning a lane etc, humans did that once today but it may be used better. Excited for the TI to see less limitation, a better shop system (lot of money throw away) and maybe better ward.
- 21 8y agoThe next AI will always be much better (see AlphaGo progression). Humans, not so much, as in all top-level competitions, human abilities improve minimally at the top, because we have millions of humans competing against each other until the plateau of human performance is reached. Then you can push that a bit more with drugs (see doping in sports). And after that, you are pretty much done. So it's only a matter of time and effort until AIs are fully unbeatable.
- douglaswlance 8y agoHumans tend to progress by learning from each other. The AI will teach us.
- asdfasgasdgasdg 8y agoWhether the AI teaches us or not, once it surpasses us, we will not catch up. This is the case for every game AI that has ever surpassed human performance so far, and there's no reason to expect that this will be different in the case of DotA.
- jlebar 8y agoIf there are any OpenAI folks hanging out here, I'd be really curious to hear about the (apparent) tactical pause by the AI in game 2.
- foobaw 8y agoMy buddy who is a pro is going to ask this during the Q&A
- mirashii 8y agoIt wasn't a bot, it was an observer. A bot would have it's name appear.
- std_throwaway 8y agoI'd speculate it's the element of surprise. Ideally the pause would do nothing to the game. (It could be that the game has some glitches but let's assume it's solid.) Then one could assume that the pause does nothing regarding the AI players. But that is not true because the AI itself is not paused (I assume) and it has to handle this unusual input. This might throw off some versions of it and it probably learns how to handle the pause and how to use it to gain a small advantage. So it's all part of the game. Similar to when a human team uses the pause for tactical purposes.
- hackandtrip 8y agoChat deciding heroes can be so cool! We may see new strategy, AI out of the safe lane confidence!!
- NegatioN 8y agoMy thoughts on the AI's performance so far knowing a bit about both DOTA and machine learning: - Item-usage for things such as smoke and wards (which were recently added to their reportoire) are not well captured by the bots yet. And the buying of wards were confirmed as a scripted event. It seems hard for them to capture the long term sparse reward of these. Smoke might not be needed by a perfect agent, but wards should be. The developer interview noted that it's not very clear what the reward for an agent warding even should be. - Some of the big advantages OpenAI gains are in team fights, where it's 200ms reaction time (upped from 80ms to resemble human reaction time more) still strikes me as something that tilts it into a solid mechanical advantage. On several occasions OpenAI's Lion managed to disable a human player performing (what I assume to be) a move which shouldn't be interruptable. (blink-->shift+ctrl ultimate ability on earthshaker) This could have tilted teamfights in the human team's favor a few times if it wasn't stopped by the "machine-like" reflexes of OpenAI. - Positioning before team fights by OpenAI are scary. There are very few openings, and every individual agent is protected by its team. - Likewise is the map movement scary most of the time. Being able to recall just enough agents back to defend while simultaneously taking out strategic objectives of the human team. Also Blitz(caster) has noted earlier OpenAI's ability to focus on the winnable lanes and sacrifice the others, prioritizing well. (and exploiting map mechanics unknown to most pros until a few years ago) - When the AI takes down a tier 1 tower, they seem to be very quick to take down the remaining tier 1 towers, instantly capitalizing on their map control advantage, and expanding it. Some interesting things / bugs: - Sniper bot throwing multiple spells on the same location right away (even though the damage doesn't stack) effectively simply wasting his mana and cooldowns, for no gain. - Sniper using his ultimate ability to pressure the lower hp characters of the human team continously. Usually it's more often used more as a finisher, and might be an artifact that appears from the AI having access to an unkillable courier that ferries a lot of healing/mana regenerating items. Some additional info learned from the interview of some of the devs: - Incentivizing killing Roshan (a boss character in the middle of the map, which yields a one-time ressurection item for one player, after being killed) is done by varying roshan's HP down to a really low amount, making sure the AI experiences the upside to this. Otherwise it would require all 5 agents gathering there, expending their magical abilities and investing a lot before actually seeing a reward. (which is unlikely to happen) - Game length in self-play sessions are above 60 minutes around 1% of the time.
- lawrenceyan 8y agoExpecting 3-0 slam dunk here. Consistently surprised at the ability of tried and true basic reinforcement learning at completing challenging tasks. I totally expected some kind of neccessary breakthrough in the RL field before a real time game like Dota 2 could be beaten.
- skgoa 8y agoThis still isn't a real game of doto, tho
- ionforce 8y agoI would love to know why the third match turned sour. I suspect (as an amateur with no ML background) that that matchup was under-trained. Like I could imagine OpenAI getting stuck in a subset of the draft pool for which it trained against, like maybe the top 10 of 18 champs. And then picking outside of that meta causes it to fall back on much less robust training/strategy.
- zackaman 8y agoBecause the first two matches were so lopsided, the bot lineup was selected by twitch chat + audience members. We drafted them a pretty terrible lineup, and from the start the bots estimated their chance of winning to be about 2.9%.
- mellinoe 8y agoIndeed. And to explain further: not all hero combinations are equal. Meaning: you cannot select any arbitrary set of 5 heroes and expect them to perform well. Different heroes have different strengths and synergies that make them stronger or weaker depending on the specific teammates and opponents that are present. This is why drafting is considered such an important (and difficult) portion of the game. In match 3, a purposefully-bad team was selected. It would have been VERY impressive if it was able to win.
- ionforce 8y ago
- vn0m 8y agoOnly one LSTM per bot so there is no "master strategy" neural network. Interesting, if I'm not wrong, bots are learning independently.
- currymj 8y agoDuring training they were given access to each other’s reward functions, and the extent to which they weighted total team reward over their own was gradually increased.
- rasz 8y agoclicked on the stream, title should be changed to "getting destroyed by Humans"
- ufo 8y agoIn the last game of the series the bots were forced to play an intentionally terrible hero lineup. The humans finally got to win a game but more interesting for me was that the times when the AI did seemingly crazy things were much more common. I wonder if this is an artifact of the training methodology: maybe if your team is very weak then your choices are also weaker, and reinforcement learning doesn't work as well?
- tux3 8y agoIt reminds me of the Go AIs going on tilt when they're far behind. When the win percentage for Go AIs gets to around 5%, every action it can take results in a losing game so it can't make the difference between normal play and super strange moves anymore. When every choice is really bad, humans tend to still go with their normal strategy and wait for their chance to turn things around, but bots assume the opponent is playing perfectly, so they act like their winrate is going to stay near zero no matter what they do.
- deleted 8y ago[deleted]
- nopinsight 8y agoIf this is true (which it might be, I haven’t studied these systems in detail), then an obvious fix is to get the AI to randomly train with ‘weaker’ versions of itself (perhaps while the strong instance is handicapped) in addition to the latest generation. Several levels of weak opponents should be used, with varying probabilities, to tune the AI’s robustness against real-world, imperfect competitors.
- dooglius 8y agoThat's probably part of it, but another explanation would be that the AI wouldn't choose such a lineup on its own, and so it doesn't train on the relevant gameplay.
- hackandtrip 8y agoI'd say that those were crazy things, but not too much. AI understood that teamfight were impossibile to win, and so the really weird choices (sven dying for the tower lol) . I'd say that matches with such lineup are not too much and so they didn't really know, but they did an awesome job and kept things balanced for so long
- yters 8y agoAI winning due to super human abilities is about as impressive as a Counter Strike bot with perfect aim making headshots every time.
- thomasahle 8y agoCheck the DeepMind Quake bot: https://deepmind.com/blog/capture-the-flag/ https://deepmind.com/blog/capture-the-flag/ Even when reducing tagging accuracy to below human level, they still performed better.
- nopinsight 8y agoWhat the cases so far tell us: Once an AI beats humans who have spent 10,000 hours practicing a skill, it is a matter of time before it beats the best professionals in the field. Cases it already happened: board games such as Chess and Go, Poker, diagnostics of certain diseases using medical images Cases where AI is still clearly inferior: video understanding, natural language understanding, motor control esp of hands and legs, general medicine, driving Hard-to-classified cases (AI is better for some instances, worse for others): image tagging and classification, speech recognition (speech-to-text), diagnostics of certain other diseases using medical images (which might need to take into account other information outside of images) More examples esp counter examples are welcome.
- hsikka 8y agoI actually have an intuition around using modular neural networks with dynamic topology to tackle more complex disease cases, general medicine, and other complex, hierarchical problems. I'm working towards trying to use them in my thesis for school
- hsikka 8y agoIf anyone has any reading suggestions or research I should look into, I'm all ears
- badpun 8y ago> Poker This is a misnomer. The only variant of Poker where AI beats humans is heads-up (two-player) variant, which is simplest form of poker and also rarely played. The AI was (marginally) beating humans there by playing a game-theory-optimal strategy. For poker games with 3+ players, the GTO strategy (Nash equilibrium) no longer exists, so AIs need to use more standard techniques (search-based, reinforcement learning etc.), which are, at the current state of the art, laughably weak at poker. Not to mention, that in poker the actual hierarchy of players' skill is not 100% obvious. You could distinguish at least two areas of skills: - play vs other experts - play vs amateurs/weaker players. Here' the goal is not to come out ahead (which, in long term, is a given), but to _maximize_ the dollar amount taken from these players, which is a skill in itself.