5 ms·

By "unbeatable" they mean it lasted a few hours in the hands of the public before the hardcoded behaviors were exploited and it was defeated consistently. Turns

by crsv 9y ago

By "unbeatable" they mean it lasted a few hours in the hands of the public before the hardcoded behaviors were exploited and it was defeated consistently. Turns out hardcoding the mechanism for players to manually block creeps (a more "creative" play style, but a fundamental element of lane control) allowed for the exploit. The bot did not teach itself to control the flow of creeps to gain a laning advantage against the opponent. Not to diminish the teams' achievement. This was awesome. But again, it's just more over-hyping of AI that should be called out for what it really is.

sambe 9y ago
They neither call it unbeatable nor mention hardcoding anything. The "unbeatable" is a quote from a professional tester, and they discuss the losses, acknowledging the need for further improvements for more general play.
- kevinwang 9y ago
  Looks like they did hardcode the creep blocking behavior. Well, they hardcoded the reward for creep blocking and trained the bot specifically to learn behavior to maximize for that reward.
  - visarga 9y ago
    ... and used curriculum learning. But that is not unfair. Humans receive plenty of curriculum training as well, we're not supposed to figure out the world by bumping into walls. Even in Dota2, the top players learned from observing each other how to deal with the bot. In fact, efficient retraining to include new strategies on the spot would be a very human-like learning ability.
    - DeepRote 9y ago
      Yeah but humans can more easily make macro-decisions based on micro situations. It's easier for us to look at a map and figure out which side is winning, sometimes that's really hard for a computer to do. I'm not a Dota2 player, but like SC2 for example is a game with LOTS of room for AI improvements. I've always thought that having some sort of APM limit might actually encourage AI authors to adopt new and unique approaches to macro-strats, but it doesn't seem to be on the horizon. When it comes to do a small thing rapidly, I think bots are almost always going to win. When it comes to do something large-scale with finesse, I think humans are going to have an advantage for a LONG time. I think that part of what makes human agents so effective at certain tasks, especially in the context of being up against another human is that we can evaluate an event and better understand the WHY of it relative to the player that played it. If I see a player pull back a bit, sometimes I think to myself that maybe they saw something they weren't expecting or something they weren't quite sure how to handle. When a computer sees the same move, a floating point number among millions changes slightly. I can try and figure out why they might be pulling back, if I did something weird or if I did something totally normal I might suspect it is bait, etc. I can think all these things in a short period of time and while large AIs might have better FLOPS than me, it doesn't understand what I'm doing, why I do it, etc. Curriculum learning isn't as effective in bots as it is in humans is my contention, I guess. Fair/unfair is a pointless observation when it comes to humans vs bots. The diversity of human-based problem solving is the perfect friction to train AIs against, imo.
  - EGreg 9y ago
    The question is whether AI would have ever learned creep blocking on its own. Or whether creep blocking is really as useful as people say. That is the key to the whole thing.
- unityByFreedom 9y ago
  > They neither call it unbeatable nor mention hardcoding anything. What about this tweet from OpenAI? "Our Dota 2 AI is undefeated against the world's best solo players" [1] Also Musk called it more complex than Go, "OpenAI first ever to defeat world's best players in competitive eSports. Vastly more complex than traditional board games like chess & Go." [2] [1] https://twitter.com/OpenAI/status/896157788908290048 https://twitter.com/OpenAI/status/896157788908290048 [2] https://twitter.com/elonmusk/status/896163163581825025?lang=en https://twitter.com/elonmusk/status/896163163581825025?lang=...
  - Chestofdraw 9y ago
    It's very misleading. It's not a Dota 2 AI it's a 1v1 SF mid AI. It's not undefeated, Suma1l and another pro beat it. I think there is a fair argument for Dota being vastly more complex than Go but there almost certainly isn't for 1v1 SF mid.
skgoa 9y ago
Yeah, the more I learn about this the less I'm impressed. Seems like they wanted to steal DeepMind's/Blizzard's thunder and hacked together a demo that isn't quite as advanced as they made it out to be.
- PeanutCurry 9y ago
  To be fair, I think this is a normal reaction to hyped AI in gaming. I remember reading an article (comment chain?) some time ago on here about how a lot of people though Deep Blue would beat Kasparov through some sort of advanced understanding of the human mind, a new sort of intuition, etc. But ultimately it came down to being able to brute force the potential move space quickly. The games will change but I wonder if the overarching effectiveness of AI victory through computational brute force will.
  - visarga 9y ago
    > being able to brute force the potential move space quickly MC sampling is an essential tool (when used together with simulation and neural nets). Don't dismiss it yet, it has a lot to offer to deep learning. For example, if you don't have enough data, you can create it by brute forcing / simulation.

by crsv 9y ago

sambe 9y ago

They neither call it unbeatable nor mention hardcoding anything. The "unbeatable" is a quote from a professional tester, and they discuss the losses, acknowledging the need for further improvements for more general play.

kevinwang 9y ago
Looks like they did hardcode the creep blocking behavior. Well, they hardcoded the reward for creep blocking and trained the bot specifically to learn behavior to maximize for that reward.
- visarga 9y ago
  ... and used curriculum learning. But that is not unfair. Humans receive plenty of curriculum training as well, we're not supposed to figure out the world by bumping into walls. Even in Dota2, the top players learned from observing each other how to deal with the bot. In fact, efficient retraining to include new strategies on the spot would be a very human-like learning ability.
  - DeepRote 9y ago
    Yeah but humans can more easily make macro-decisions based on micro situations. It's easier for us to look at a map and figure out which side is winning, sometimes that's really hard for a computer to do. I'm not a Dota2 player, but like SC2 for example is a game with LOTS of room for AI improvements. I've always thought that having some sort of APM limit might actually encourage AI authors to adopt new and unique approaches to macro-strats, but it doesn't seem to be on the horizon. When it comes to do a small thing rapidly, I think bots are almost always going to win. When it comes to do something large-scale with finesse, I think humans are going to have an advantage for a LONG time. I think that part of what makes human agents so effective at certain tasks, especially in the context of being up against another human is that we can evaluate an event and better understand the WHY of it relative to the player that played it. If I see a player pull back a bit, sometimes I think to myself that maybe they saw something they weren't expecting or something they weren't quite sure how to handle. When a computer sees the same move, a floating point number among millions changes slightly. I can try and figure out why they might be pulling back, if I did something weird or if I did something totally normal I might suspect it is bait, etc. I can think all these things in a short period of time and while large AIs might have better FLOPS than me, it doesn't understand what I'm doing, why I do it, etc. Curriculum learning isn't as effective in bots as it is in humans is my contention, I guess. Fair/unfair is a pointless observation when it comes to humans vs bots. The diversity of human-based problem solving is the perfect friction to train AIs against, imo.
- EGreg 9y ago
  The question is whether AI would have ever learned creep blocking on its own. Or whether creep blocking is really as useful as people say. That is the key to the whole thing.
unityByFreedom 9y ago
> They neither call it unbeatable nor mention hardcoding anything. What about this tweet from OpenAI? "Our Dota 2 AI is undefeated against the world's best solo players" [1] Also Musk called it more complex than Go, "OpenAI first ever to defeat world's best players in competitive eSports. Vastly more complex than traditional board games like chess & Go." [2] [1] https://twitter.com/OpenAI/status/896157788908290048 https://twitter.com/OpenAI/status/896157788908290048 [2] https://twitter.com/elonmusk/status/896163163581825025?lang=en https://twitter.com/elonmusk/status/896163163581825025?lang=...
- Chestofdraw 9y ago
  It's very misleading. It's not a Dota 2 AI it's a 1v1 SF mid AI. It's not undefeated, Suma1l and another pro beat it. I think there is a fair argument for Dota being vastly more complex than Go but there almost certainly isn't for 1v1 SF mid.

skgoa 9y ago

Yeah, the more I learn about this the less I'm impressed. Seems like they wanted to steal DeepMind's/Blizzard's thunder and hacked together a demo that isn't quite as advanced as they made it out to be.

PeanutCurry 9y ago
To be fair, I think this is a normal reaction to hyped AI in gaming. I remember reading an article (comment chain?) some time ago on here about how a lot of people though Deep Blue would beat Kasparov through some sort of advanced understanding of the human mind, a new sort of intuition, etc. But ultimately it came down to being able to brute force the potential move space quickly. The games will change but I wonder if the overarching effectiveness of AI victory through computational brute force will.
- visarga 9y ago
  > being able to brute force the potential move space quickly MC sampling is an essential tool (when used together with simulation and neural nets). Don't dismiss it yet, it has a lot to offer to deep learning. For example, if you don't have enough data, you can create it by brute forcing / simulation.