11 ms·

OpenAI’s Dota 2 defeat is still a win for artificial intelligence

59 points by dsr12 8y ago

chaoz_ 8y ago
Typical title in this AI-Hype time period. Group of people came up with a model and were able to compete with nearly unlimited training data. Wow. Don't think match win would really mean something. Only from the marketing perspective.
- kamens 8y ago
  Sometimes it feels like there's a secret competition to see who can author the next infamous "No wireless. Less space than a nomad. Lame."-style comment.
  - chongli 8y ago
    I saw that comment when it was first posted on Slashdot all those years ago. The original iPod was overpriced and it only worked on Macs with FireWire. It really didn't take off until they had a USB model working on Windows at a more reasonable price.
ir193 8y ago
Instead of coding the bots with the rules of Dota 2, they’re thrown into the game and left to figure things out for themselves I saw a strange behaviour. In the second game, I saw witch doctor use "Maledict" on neutral creeps, but this skill only affects enemy heroes. This only waste mana, put the skill in cool down and have no benefit. How can AI learn it?
- BOBOTWINSTON 8y ago
  Maybe he was baiting, next level strats.
- deleted 8y ago
  [deleted]
- SolarNet 8y ago
  Because neural networks are noisy, and they cause small mistakes and misunderstandings like in humans. The AIs concept of an enemy obviously doesn't have as strong of a distinction between "hero" and "creep" (and for that matter "enemy" and "neutral") as those of us who have read the game rules might have because of your quoted statement. As a further analogy I often see humans make ridiculous and insane leaps of logic (how assignment works when you first learn programming; free speech applies to things besides governments; etc.), and we run on the same style of hardware. To an extent this demonstrates how far we are from generalized AI. All the Open AI system is is a very specialized animal, that's lived through millions of simulated generations very quickly. It doesn't understand language or concepts - we can't even begin to tell it the rules of the game, only though simulated evolution can we teach it - or even the self awareness many animals do. It's likely closer to the nervous system of an insect than it is to the brain of a dog.
FartyMcFarter 8y ago
Good article! > First, OpenAI could have created a vision system to read the pixels and retrieve the same information that the bot API provides. (The main reason it didn’t is that it would have been incredibly resource-intensive.) Mmmm, no. Reading the pixels would not retrieve the same information that the bot API provides, because the screen does not contain all that information at any given time. Any Dota 2 player will know that they don't have access to all this data simultaneously: https://s3-us-west-2.amazonaws.com/openai-assets/dota_benchmark_results/network_diagram_08_06_2018.pdf https://s3-us-west-2.amazonaws.com/openai-assets/dota_benchm...
- tlb 8y ago
  It would have to remember the data that is off-screen. It's quite possible to train a recurrent network to do this. Some Atari games require this to play well, like Pacman requires remembering where the ghosts are when they're blinking. More complicated is the logic to know what to look at when, by keeping track of which parts of the state might be important and out of date, and controlling the viewpoint to update it. I haven't seen a good example of this.
  - FartyMcFarter 8y ago
    True... well, almost, since data from previous frames becomes stale with time. I agree that learning how to retrieve the info would be another rather tough problem (on top of all the tough problems the current incarnation of OpenAI's bots already need to solve).
- SolarNet 8y ago
  Yes, but then after reading data, they could train a system to remember the information, and decide which information to look at next, and predict their values over time from what inputs it gets, like humans do.
- empath75 8y ago
  I think the reason they didn't do it is that image recognition is a different and less interesting problem than the one they're trying to solve.
cowmoo728 8y ago
Isn't it the case that many behaviors had to be specifically encouraged or coded into the bots to get them to work at all? They learned an impressive amount on their own after being pushed in the right direction, but this is far from a pure random reinforcement learning approach as this article suggests. This article also really understates how clowny the bots were playing. They appeared confused and aimless for much of the game, until they found a clear objective like an enemy hero that moved too close to them, or they had a numbers advantage that allowed them to pressure an enemy building. They were brutal at closing gaps and pursuing enemies to kill without hesitation, but didn't perform well at all beyond that. This isn't to take away from the accomplishments of OpenAI - to get to this stage after 18 months of work is an impressive engineering feat. It's just telling that the bots have only learned one single strategy, don't even execute that very well, and still have to play with hero restrictions that rule out counters to that specific strategy of aggressive skirmishing and pushing.
- tlb 8y ago
  Looking "clowny" isn't something that RL agents learn to avoid. You see it even in Go agents. The MCTS algorithm looks at probability of winning, and if it's ~100% regardless of the next move, it'll often waste moves rather than trying to win as quickly as possible.
- SOLAR_FIELDS 8y ago
  My understanding is that there are behaviors that are coded into the bots at first to bootstrap their learning capabilities, which are then removed after a certain amount of learning has taken place. In another article that was posted here a few days ago one of the OpenAI team talks about removing the last of these "scripted" capabilities as part of the improvements they plan on making to the bots after this round of games against human players. I kind of liken it to a human who learns math - you start off by learning the rules (addition, subtraction, algebra, etc.) and work within those confines before you start to use these basic rules to push the boundaries and explore (proofs).
- dx87 8y ago
  Some behaviors did have to be encouraged, but from what I understand, the behaviors were encouraged through small indirect nudges instead of specifically adding code to the bots to tell them a certain behavior is good. For example, the devs said that the bots never tried to fight Roshan on their own because he does a lot of damage, has a lot of health, and the bots don't know what the reward for killing him is. To encourage the bots to try it, Roshan was given artificially low health so the bots would kill it to see what happens. Once the bots learned that there are rewards for killing him, the devs started to randomize his health between very high and very low values so the bots could get a feel for when they are strong enough to try and kill him during a normal match.
acroback 8y ago
How is this monstrosity going to understand that it is not just practice which makes humans good at team work in Dota2? AI has long ways to go before it can defeat humans in a complex game like Dota2.
- ggggtez 8y ago
  Well the point is that practice actually is enough. Humans have a tendency to think they are special because they can do X. Then a robot does X, and suddenly the goal posts move. Humans are just specialized neural networks plus meat.
  - yters 8y ago
    Once AI can move the goalposts, then I'll be worried.
- dragontamer 8y ago
  Frankly, superior reaction time and other mechanical skills in Dota2 (or Starcraft, or other games which are being tested) makes me think that a typical AI would beat a human. https://www.youtube.com/watch?v=IKVFZ28ybQs https://www.youtube.com/watch?v=IKVFZ28ybQs That's the "gloves come off" AI. Granted, its a problem solved by raw, mechanical button-pushes at a rate beyond what is possible on a keyboard/mouse, but it demonstrates the greatest skills an AI or computer has. Of course, no one wants to see an AI beat a human using known methodologies. They want to see an AI win at the abstract game of "a game of strategy". From this perspective, I think the poker AI results over the past year or two have been far more interesting. AIs figuring out the Nash Equalibrium and playing bluffing games around it. Sure, its a bit mechanical and mathematically inclined, but generalizing and solving the bluffing game is IMO far more useful.
guskel 8y ago
Humans still beat machines at most of the Atari 2600 games. https://github.com/cshenton/atari-leaderboard https://github.com/cshenton/atari-leaderboard And that list is incomplete if you want to include human high scores on non-emulators (not included in that list). This is even after the reinforcement learning algorithms have been given orders of magnitude more training time than humans. Furthermore, much of the machine performance over humans can be attributed to better reaction time.
- yazr 8y ago
  You are incorrect. IIRC, you can easily build a super-human Atari bot. Just add some off line planning (e.g. paper by Guo), or add manual rewards or features. No one is doing since there is no point. We want to study Atari WITHOUT these "cheats", so that we can then apply these algos in more complex situations.
  - daveguy 8y ago
    I think the OP was using "machines" as reinforcement learning (or whatever the best general learning algorithm). It doesn't count if you program a solution for each individual game. These are learning algorithm benchmarks not programmer benchmarks. For generalized reinforcement learning algorithms. Humans almost always beat the "machine". The only cases where machine wins is on 100% hand eye coordination and duration. All ML algorithms are terrible at figuring out how to plan de-novo. So anything that requires multiple contexts or planning is a fail.
    - guskel 8y ago
      Yes, my intent was referring to reinforcement learning algorithms. Writing a handcrafted program to beat an arbitrary Atari game is trivial. Yet on the other hand, it is apparent that reinforcement learning algorithms have surpassed humans in board games such as Go and Chess.
YeGoblynQueenne 8y ago
>> The more important question might be: can we ever have a fair fight between humans and machines? Fair or not fair, there is no API to the real world. If we are going to create machines that can learn and think outside of simulations, however complex they may be, these machines will need a way to interact with the physical environment. Gary Marcus is on the right track here. Building systems that can "handle the complexity of the real world", as per OpenAI's stated goal according to the article, is incredibly ambitious and pointing out the vast distance separating the current state-of-the-art from that lofty goal is, well, only fair. I mean, if you think about it, back in the '70s, in the original AI Winter, one of the big criticisms of AI research was that it languished in simulated environments like blocks world and didn't perform nearly as well in the real world. And here we are today, celebrating a bright step on the path to conquering yet another simulation.
- piyh 8y ago
  Our simulations have gotten way better in 40 years. You could teach an AI the basics of simulation driving or flying that would work as a baseline for the real world.
  - YeGoblynQueenne 8y ago
    Unfortunately, modern, statistical machine-learning based AI is extremely bad at generalising from one environment or one context to another. And, training one some task in a simulation and then performing the same task in the real world requires a very strong ability for generalisation. That is because it is extremely expensive to simulate the real world with any fidelity and therefore every simulation is "cutting corners" - and really, really big corners at that. This gap, between reality and simulation must be covered by generalisation, but our machine learning systems genearlise too poorly. As a result, training in simulated environments doesn't help handle the full complexity of the real world. Even if we had robot bodies that could move as freely and manipulate objects with as much dexterity as they can do in simulations.
    - Isinlor 8y ago
aeosynth 8y ago
Anyone know what the 'undiscovered mechanic' is? > At least one previously undiscovered game mechanic, which allows players to recharge a certain weapon quickly by staying out of range of the enemy, has been discovered by the bots and passed on to humans I assume they're talking about blink dagger, and something more advanced than "don't take damage"?
- baobolus 8y ago
  that's a bit misleading. it was discovered that if you stay out of vision and cast raze, the other player does not get stick charges. that was the 1v1 shadow fiend bot a year ago though.
  - newsoundwave 8y ago
    If that's the case, that's incredibly misleading. Especially because that's not undiscovered by human players, just by the OpenAI team, if that's what it is: https://blog.openai.com/more-on-dota-2/ https://blog.openai.com/more-on-dota-2/ "Sumail pointed out that the bot had learned to cast razes out of the enemy’s vision. This was due to a mechanic we hadn’t known about: abilities cast outside of the enemy’s vision prevent the enemy from gaining a wand charge." That's not an undiscovered mechanic in the world of Dota - that's been known for a while and at least documented since 2015 https://dota2.gamepedia.com/index.php?title=Magic_Stick&oldid=896457 https://dota2.gamepedia.com/index.php?title=Magic_Stick&oldi.... I wouldn't be surprised if it was known before then, I certainly remember this from a while back. Again, if that's not what it was, then I take back what I said, but if it was, I do think that statement is misleading as written.
  - cowmoo728 8y ago
    That mechanic was in the patch notes for dota 6.69 released in 2010. Maybe its importance to high level 1v1 shadow fiend was revealed but that's stretching it. It is cool that the bot rediscovered it though.
  - jaccarmac 8y ago
    Doubt that's what they are talking about if the mechanic was undiscovered before this year. Stick + vision mechanics have been well known for a while, I learned about it when I read about the Kunkka mid matchup. Might have been Pajkatt buying mangoes and using them out of vision last year? Not sure, my memory of the details is fuzzy.
YeGoblynQueenne 8y ago
>> Such a thing does not exist because machines think like humans in the same way that planes fly like birds. Could we please lay this empty platitude to reast? "Planes don't fly by flapping their wings like birds, so why should computers think like humans?". Well, except that we were only able to built flying machines (rather than floating ones) when we figured out why birds can fly [1]. And then, the whole of our computer science is based on the idea that a computer is, actually, the same kind of system as a human mind- a computational device, a machine that can compute everything and anything that can be computed. This is the deep insight that informs the ambition to create artificial intelligence: that brains are a kind of computer, computers are a kind of brain, and they can both compute the same kind of program, in other words- intelligence. Though we may not know how human minds work exactly and therefore we can't readily copy them, it is thanks to Turing's and Church's insight that we even have computers today. And so, in a very real sense, we can only ever create thinking machines that do intelligence in the same way that humans do intelligence [2]. _______________________ [1] I mean, a boomerang is really an airfoil so I guess Australian Aborigines had figured out flight long before the Wright brothers, but I gues they didn't need flying vehicles back then? [2] Unless there is a paradigm shift and we figure out a better way to do it etc etc disclaimer disclaimer.
- jimbokun 8y ago
  Brains are not computers, and brains were not the model for computers. Formal logic and the Turing machine were the model for computers. Turing and Church did not invent computers based on their careful study of human biology and physiology.
  - jorgemf 8y ago
    Neural networks are based on human brains. So, how are they both different? It is a very hard question, because if we would understand better the brain we will simulate it better in the computers, how do we know we are not already doing a good simulation of the brain? Any answer to this question would be a step forward to improve our artificial intelligence systems.
  - YeGoblynQueenne 8y ago
YeGoblynQueenne 8y ago
>> Already, the training infrastructure used to teach the OpenAI Five — a system called Rapid — is being turned to other projects. OpenAI has used it to teach robot hands to manipulate objects with new levels of human-like dexterity, for example. As always with AI, there are limitations, and Rapid isn’t some do-everything algorithm. Oh, yes, indeed, there are limitations. The robot hand in question can only manipulate cubes and then only a specific kind of cube with standard dimensions, as far as it's possible to tell from all the demonstrations publicised by OpenAI. And, I'm guessing, if they had a robot hand able to play the yo-yo, they wouldn't hesitate to show it.
DrNuke 8y ago
These are the latest wargames instalments (you know, the Matthew Broderick kool-aid from his 1983 film and so on) and still in their infancy for optimising local, transient tactics behind the overall goal. AIs playing Civilization by Sid Meier vs themselves is fascinating enough and possibly useful for transfer learning here? At the very least, the weakest links in real mid-game situations coming from learned-led plans would be discovered. Apart of military uses of such studies and many other techniques giant corps masquerade behind emoticons and facial recognition apps, why not let the latest AIs play vs climate change, for example? It would be much more fun in general and possibly able to highlight the contribution of many impending agents humans can’t fathom already.
kenhwang 8y ago
As someone who played a bunch of Dota2 (and was present at TI where this happened), the bots were a joke -- and the humans that played against them were not only super sandbagging and screwing around, they quickly figured out the bots' preferred strategy (deathball) and came up with an effective counterstrategy (split push) that the bots responded very poorly to. The bots excelled at things you expected computers to be good at, consistently quick reactions/mechanical skill and coordinated skirmishes. The bots fell flat on strategy, optimal resource usage (short and long term), and most importantly, coordinated decision-making. So as an result, they dominated the early game which has always been heavily mechanics favored and absolutely fell apart mid/late-game when decision-making mattered (and often made decisions that benefited early game economy at the cost of mid/late game). Computers being good at mechanics is not an impressive result.
- whymauri 8y ago
  Calling them a 'joke' is pretty extreme. Did you watch the benchmark? They were pretty much steamrolling the caster team. With all due respect, the professionals who have played against the bot have been pretty impressed with previous iteration. The major roadblocks at TI were probably that the bot did not efficiently learn courier mechanics and that the draft was not done by the bot.
  - foota 8y ago
    What about parents comments about the bots deficits? It doesn't really matter if the bots win if they can only win due to mechanical advantage imo.
    - whymauri 8y ago
      There is no mechanical advantage, they have their response time capped at 200ms which is around an elite player's reaction. They can definitely strategize for ganks and the future reward function anticipates high reward roaming/ganking plays.
      - 8y ago
knbknb 8y ago
Was the full list of heroes supported in theses showmatches? Or was this also restricted to a subset of, say, 15 out of 150 (or whatever that number is)? The announcers/casters were a bit ambiguous about this, AFAIK they said something like : "the teams agreed on a predefined, balanced list of heroes before the game...", and there was no drafting. (I'm not native speaker, and I don't know much about Dota2 rules)
- Master_Odin 8y ago
  It was limited to a pool of 18 heroes still.
fabiofzero 8y ago
The headline reads suspiciously like the legendary "Bitcoin has crashed: here's why this is a good thing"

59 points by dsr12 8y ago

Typical title in this AI-Hype time period. Group of people came up with a model and were able to compete with nearly unlimited training data. Wow. Don't think match win would really mean something. Only from the marketing perspective.

kamens 8y ago
Sometimes it feels like there's a secret competition to see who can author the next infamous "No wireless. Less space than a nomad. Lame."-style comment.
- chongli 8y ago
  I saw that comment when it was first posted on Slashdot all those years ago. The original iPod was overpriced and it only worked on Macs with FireWire. It really didn't take off until they had a USB model working on Windows at a more reasonable price.

ir193 8y ago

Instead of coding the bots with the rules of Dota 2, they’re thrown into the game and left to figure things out for themselves I saw a strange behaviour. In the second game, I saw witch doctor use "Maledict" on neutral creeps, but this skill only affects enemy heroes. This only waste mana, put the skill in cool down and have no benefit. How can AI learn it?

BOBOTWINSTON 8y ago
Maybe he was baiting, next level strats.
deleted 8y ago
[deleted]
SolarNet 8y ago
Because neural networks are noisy, and they cause small mistakes and misunderstandings like in humans. The AIs concept of an enemy obviously doesn't have as strong of a distinction between "hero" and "creep" (and for that matter "enemy" and "neutral") as those of us who have read the game rules might have because of your quoted statement. As a further analogy I often see humans make ridiculous and insane leaps of logic (how assignment works when you first learn programming; free speech applies to things besides governments; etc.), and we run on the same style of hardware. To an extent this demonstrates how far we are from generalized AI. All the Open AI system is is a very specialized animal, that's lived through millions of simulated generations very quickly. It doesn't understand language or concepts - we can't even begin to tell it the rules of the game, only though simulated evolution can we teach it - or even the self awareness many animals do. It's likely closer to the nervous system of an insect than it is to the brain of a dog.

FartyMcFarter 8y ago

Good article! > First, OpenAI could have created a vision system to read the pixels and retrieve the same information that the bot API provides. (The main reason it didn’t is that it would have been incredibly resource-intensive.) Mmmm, no. Reading the pixels would not retrieve the same information that the bot API provides, because the screen does not contain all that information at any given time. Any Dota 2 player will know that they don't have access to all this data simultaneously: https://s3-us-west-2.amazonaws.com/openai-assets/dota_benchmark_results/network_diagram_08_06_2018.pdf https://s3-us-west-2.amazonaws.com/openai-assets/dota_benchm...

tlb 8y ago
It would have to remember the data that is off-screen. It's quite possible to train a recurrent network to do this. Some Atari games require this to play well, like Pacman requires remembering where the ghosts are when they're blinking. More complicated is the logic to know what to look at when, by keeping track of which parts of the state might be important and out of date, and controlling the viewpoint to update it. I haven't seen a good example of this.
- FartyMcFarter 8y ago
  True... well, almost, since data from previous frames becomes stale with time. I agree that learning how to retrieve the info would be another rather tough problem (on top of all the tough problems the current incarnation of OpenAI's bots already need to solve).
SolarNet 8y ago
Yes, but then after reading data, they could train a system to remember the information, and decide which information to look at next, and predict their values over time from what inputs it gets, like humans do.
empath75 8y ago
I think the reason they didn't do it is that image recognition is a different and less interesting problem than the one they're trying to solve.

cowmoo728 8y ago

Isn't it the case that many behaviors had to be specifically encouraged or coded into the bots to get them to work at all? They learned an impressive amount on their own after being pushed in the right direction, but this is far from a pure random reinforcement learning approach as this article suggests. This article also really understates how clowny the bots were playing. They appeared confused and aimless for much of the game, until they found a clear objective like an enemy hero that moved too close to them, or they had a numbers advantage that allowed them to pressure an enemy building. They were brutal at closing gaps and pursuing enemies to kill without hesitation, but didn't perform well at all beyond that. This isn't to take away from the accomplishments of OpenAI - to get to this stage after 18 months of work is an impressive engineering feat. It's just telling that the bots have only learned one single strategy, don't even execute that very well, and still have to play with hero restrictions that rule out counters to that specific strategy of aggressive skirmishing and pushing.

tlb 8y ago
Looking "clowny" isn't something that RL agents learn to avoid. You see it even in Go agents. The MCTS algorithm looks at probability of winning, and if it's ~100% regardless of the next move, it'll often waste moves rather than trying to win as quickly as possible.
SOLAR_FIELDS 8y ago
My understanding is that there are behaviors that are coded into the bots at first to bootstrap their learning capabilities, which are then removed after a certain amount of learning has taken place. In another article that was posted here a few days ago one of the OpenAI team talks about removing the last of these "scripted" capabilities as part of the improvements they plan on making to the bots after this round of games against human players. I kind of liken it to a human who learns math - you start off by learning the rules (addition, subtraction, algebra, etc.) and work within those confines before you start to use these basic rules to push the boundaries and explore (proofs).
dx87 8y ago
Some behaviors did have to be encouraged, but from what I understand, the behaviors were encouraged through small indirect nudges instead of specifically adding code to the bots to tell them a certain behavior is good. For example, the devs said that the bots never tried to fight Roshan on their own because he does a lot of damage, has a lot of health, and the bots don't know what the reward for killing him is. To encourage the bots to try it, Roshan was given artificially low health so the bots would kill it to see what happens. Once the bots learned that there are rewards for killing him, the devs started to randomize his health between very high and very low values so the bots could get a feel for when they are strong enough to try and kill him during a normal match.

acroback 8y ago

How is this monstrosity going to understand that it is not just practice which makes humans good at team work in Dota2? AI has long ways to go before it can defeat humans in a complex game like Dota2.

ggggtez 8y ago
Well the point is that practice actually is enough. Humans have a tendency to think they are special because they can do X. Then a robot does X, and suddenly the goal posts move. Humans are just specialized neural networks plus meat.
- yters 8y ago
  Once AI can move the goalposts, then I'll be worried.
dragontamer 8y ago
Frankly, superior reaction time and other mechanical skills in Dota2 (or Starcraft, or other games which are being tested) makes me think that a typical AI would beat a human. https://www.youtube.com/watch?v=IKVFZ28ybQs https://www.youtube.com/watch?v=IKVFZ28ybQs That's the "gloves come off" AI. Granted, its a problem solved by raw, mechanical button-pushes at a rate beyond what is possible on a keyboard/mouse, but it demonstrates the greatest skills an AI or computer has. Of course, no one wants to see an AI beat a human using known methodologies. They want to see an AI win at the abstract game of "a game of strategy". From this perspective, I think the poker AI results over the past year or two have been far more interesting. AIs figuring out the Nash Equalibrium and playing bluffing games around it. Sure, its a bit mechanical and mathematically inclined, but generalizing and solving the bluffing game is IMO far more useful.

guskel 8y ago

Humans still beat machines at most of the Atari 2600 games. https://github.com/cshenton/atari-leaderboard https://github.com/cshenton/atari-leaderboard And that list is incomplete if you want to include human high scores on non-emulators (not included in that list). This is even after the reinforcement learning algorithms have been given orders of magnitude more training time than humans. Furthermore, much of the machine performance over humans can be attributed to better reaction time.

yazr 8y ago
You are incorrect. IIRC, you can easily build a super-human Atari bot. Just add some off line planning (e.g. paper by Guo), or add manual rewards or features. No one is doing since there is no point. We want to study Atari WITHOUT these "cheats", so that we can then apply these algos in more complex situations.
- daveguy 8y ago
  I think the OP was using "machines" as reinforcement learning (or whatever the best general learning algorithm). It doesn't count if you program a solution for each individual game. These are learning algorithm benchmarks not programmer benchmarks. For generalized reinforcement learning algorithms. Humans almost always beat the "machine". The only cases where machine wins is on 100% hand eye coordination and duration. All ML algorithms are terrible at figuring out how to plan de-novo. So anything that requires multiple contexts or planning is a fail.
  - guskel 8y ago
    Yes, my intent was referring to reinforcement learning algorithms. Writing a handcrafted program to beat an arbitrary Atari game is trivial. Yet on the other hand, it is apparent that reinforcement learning algorithms have surpassed humans in board games such as Go and Chess.

YeGoblynQueenne 8y ago

>> The more important question might be: can we ever have a fair fight between humans and machines? Fair or not fair, there is no API to the real world. If we are going to create machines that can learn and think outside of simulations, however complex they may be, these machines will need a way to interact with the physical environment. Gary Marcus is on the right track here. Building systems that can "handle the complexity of the real world", as per OpenAI's stated goal according to the article, is incredibly ambitious and pointing out the vast distance separating the current state-of-the-art from that lofty goal is, well, only fair. I mean, if you think about it, back in the '70s, in the original AI Winter, one of the big criticisms of AI research was that it languished in simulated environments like blocks world and didn't perform nearly as well in the real world. And here we are today, celebrating a bright step on the path to conquering yet another simulation.

piyh 8y ago
Our simulations have gotten way better in 40 years. You could teach an AI the basics of simulation driving or flying that would work as a baseline for the real world.
- YeGoblynQueenne 8y ago
  Unfortunately, modern, statistical machine-learning based AI is extremely bad at generalising from one environment or one context to another. And, training one some task in a simulation and then performing the same task in the real world requires a very strong ability for generalisation. That is because it is extremely expensive to simulate the real world with any fidelity and therefore every simulation is "cutting corners" - and really, really big corners at that. This gap, between reality and simulation must be covered by generalisation, but our machine learning systems genearlise too poorly. As a result, training in simulated environments doesn't help handle the full complexity of the real world. Even if we had robot bodies that could move as freely and manipulate objects with as much dexterity as they can do in simulations.
  - Isinlor 8y ago

aeosynth 8y ago

Anyone know what the 'undiscovered mechanic' is? > At least one previously undiscovered game mechanic, which allows players to recharge a certain weapon quickly by staying out of range of the enemy, has been discovered by the bots and passed on to humans I assume they're talking about blink dagger, and something more advanced than "don't take damage"?

baobolus 8y ago
that's a bit misleading. it was discovered that if you stay out of vision and cast raze, the other player does not get stick charges. that was the 1v1 shadow fiend bot a year ago though.
- newsoundwave 8y ago
  If that's the case, that's incredibly misleading. Especially because that's not undiscovered by human players, just by the OpenAI team, if that's what it is: https://blog.openai.com/more-on-dota-2/ https://blog.openai.com/more-on-dota-2/ "Sumail pointed out that the bot had learned to cast razes out of the enemy’s vision. This was due to a mechanic we hadn’t known about: abilities cast outside of the enemy’s vision prevent the enemy from gaining a wand charge." That's not an undiscovered mechanic in the world of Dota - that's been known for a while and at least documented since 2015 https://dota2.gamepedia.com/index.php?title=Magic_Stick&oldid=896457 https://dota2.gamepedia.com/index.php?title=Magic_Stick&oldi.... I wouldn't be surprised if it was known before then, I certainly remember this from a while back. Again, if that's not what it was, then I take back what I said, but if it was, I do think that statement is misleading as written.
- cowmoo728 8y ago
  That mechanic was in the patch notes for dota 6.69 released in 2010. Maybe its importance to high level 1v1 shadow fiend was revealed but that's stretching it. It is cool that the bot rediscovered it though.
- jaccarmac 8y ago
  Doubt that's what they are talking about if the mechanic was undiscovered before this year. Stick + vision mechanics have been well known for a while, I learned about it when I read about the Kunkka mid matchup. Might have been Pajkatt buying mangoes and using them out of vision last year? Not sure, my memory of the details is fuzzy.

YeGoblynQueenne 8y ago

>> Such a thing does not exist because machines think like humans in the same way that planes fly like birds. Could we please lay this empty platitude to reast? "Planes don't fly by flapping their wings like birds, so why should computers think like humans?". Well, except that we were only able to built flying machines (rather than floating ones) when we figured out why birds can fly [1]. And then, the whole of our computer science is based on the idea that a computer is, actually, the same kind of system as a human mind- a computational device, a machine that can compute everything and anything that can be computed. This is the deep insight that informs the ambition to create artificial intelligence: that brains are a kind of computer, computers are a kind of brain, and they can both compute the same kind of program, in other words- intelligence. Though we may not know how human minds work exactly and therefore we can't readily copy them, it is thanks to Turing's and Church's insight that we even have computers today. And so, in a very real sense, we can only ever create thinking machines that do intelligence in the same way that humans do intelligence [2]. _______________________ [1] I mean, a boomerang is really an airfoil so I guess Australian Aborigines had figured out flight long before the Wright brothers, but I gues they didn't need flying vehicles back then? [2] Unless there is a paradigm shift and we figure out a better way to do it etc etc disclaimer disclaimer.

jimbokun 8y ago
Brains are not computers, and brains were not the model for computers. Formal logic and the Turing machine were the model for computers. Turing and Church did not invent computers based on their careful study of human biology and physiology.
- jorgemf 8y ago
  Neural networks are based on human brains. So, how are they both different? It is a very hard question, because if we would understand better the brain we will simulate it better in the computers, how do we know we are not already doing a good simulation of the brain? Any answer to this question would be a step forward to improve our artificial intelligence systems.
- YeGoblynQueenne 8y ago

YeGoblynQueenne 8y ago

>> Already, the training infrastructure used to teach the OpenAI Five — a system called Rapid — is being turned to other projects. OpenAI has used it to teach robot hands to manipulate objects with new levels of human-like dexterity, for example. As always with AI, there are limitations, and Rapid isn’t some do-everything algorithm. Oh, yes, indeed, there are limitations. The robot hand in question can only manipulate cubes and then only a specific kind of cube with standard dimensions, as far as it's possible to tell from all the demonstrations publicised by OpenAI. And, I'm guessing, if they had a robot hand able to play the yo-yo, they wouldn't hesitate to show it.

DrNuke 8y ago

These are the latest wargames instalments (you know, the Matthew Broderick kool-aid from his 1983 film and so on) and still in their infancy for optimising local, transient tactics behind the overall goal. AIs playing Civilization by Sid Meier vs themselves is fascinating enough and possibly useful for transfer learning here? At the very least, the weakest links in real mid-game situations coming from learned-led plans would be discovered. Apart of military uses of such studies and many other techniques giant corps masquerade behind emoticons and facial recognition apps, why not let the latest AIs play vs climate change, for example? It would be much more fun in general and possibly able to highlight the contribution of many impending agents humans can’t fathom already.

kenhwang 8y ago

As someone who played a bunch of Dota2 (and was present at TI where this happened), the bots were a joke -- and the humans that played against them were not only super sandbagging and screwing around, they quickly figured out the bots' preferred strategy (deathball) and came up with an effective counterstrategy (split push) that the bots responded very poorly to. The bots excelled at things you expected computers to be good at, consistently quick reactions/mechanical skill and coordinated skirmishes. The bots fell flat on strategy, optimal resource usage (short and long term), and most importantly, coordinated decision-making. So as an result, they dominated the early game which has always been heavily mechanics favored and absolutely fell apart mid/late-game when decision-making mattered (and often made decisions that benefited early game economy at the cost of mid/late game). Computers being good at mechanics is not an impressive result.

whymauri 8y ago
Calling them a 'joke' is pretty extreme. Did you watch the benchmark? They were pretty much steamrolling the caster team. With all due respect, the professionals who have played against the bot have been pretty impressed with previous iteration. The major roadblocks at TI were probably that the bot did not efficiently learn courier mechanics and that the draft was not done by the bot.
- foota 8y ago
  What about parents comments about the bots deficits? It doesn't really matter if the bots win if they can only win due to mechanical advantage imo.
  - whymauri 8y ago
    There is no mechanical advantage, they have their response time capped at 200ms which is around an elite player's reaction. They can definitely strategize for ganks and the future reward function anticipates high reward roaming/ganking plays.
    - 8y ago

knbknb 8y ago

Was the full list of heroes supported in theses showmatches? Or was this also restricted to a subset of, say, 15 out of 150 (or whatever that number is)? The announcers/casters were a bit ambiguous about this, AFAIK they said something like : "the teams agreed on a predefined, balanced list of heroes before the game...", and there was no drafting. (I'm not native speaker, and I don't know much about Dota2 rules)

Master_Odin 8y ago
It was limited to a pool of 18 heroes still.

fabiofzero 8y ago

The headline reads suspiciously like the legendary "Bitcoin has crashed: here's why this is a good thing"