6 ms·

I mean yeah, this version of the bot had drastically less time to train. Only 6 days compared to a few weeks for that version. I know how the game works, I ment

I mean yeah, this version of the bot had drastically less time to train. Only 6 days compared to a few weeks for that version. I know how the game works, I mentioned the couriers already. I 100% disagree that the invincible couriers make the game "very similar" to the 1v1. There's many more layers of engineering that went into that version. The bots had to learn drafting, cooperation, highground pushing, Roshan, vision, invisibility, etc. I'm amazed you can trivialize all this as a 'joke'.

alper111 8y ago
If you look at network structure, it acts as one agent, not five. So, free coordination. (See: https://t.co/GPKHPsIu1C https://t.co/GPKHPsIu1C) In my opinion, what i see is a very good player who knows how to chain stun precisely without any strategic depth. If you claim to have built an AI system, which you ultimately want it to evolve to AGI, you at least expect some sort of strategic decision making at the macro level. Though since it has almost perfect micro, it can easily outweight the most of teams. So yeah, with that expectation I see this as a joke, too. P.S. The model is trained with 128k cpus and 256 gpu. It is able to play 180 years worth of game in a day. Think about it.
- whymauri 8y ago
  It's five independent agents. The article on OpenAI's website and the network structure both say this. I'll zoom in since it's a complicated structure. It's the first line of the article: https://blog.openai.com/openai-five/ https://blog.openai.com/openai-five/ >Our team of five neural networks, They use a hyperparameter called team spirit to cooperate. I don't think the goal of this is AGI at all, so I don't see why people are making that leap. But sure, for the geniuses of HN this must clearly be trivial.
  - backpropaganda 8y ago
    It's not independent agents. The neural networks have the same input, share weights, and also share some activations. With that much sharing, it's better to think of it as one neural network which has output heads for all the 5 players. So coordination is free. Actually, coordination makes as little sense as saying that multiple neurons in a neural network are cooperating, or that the two legs of a humanoid are cooperating to walk. Further there is no game being played between heroes of the same team. They literally have the same objective. The "coordination" buzzword is just another attempt by OpenAI to confuse and mislead readers, and give a false sense of their progress.
    - whymauri 8y ago
      They cannot share the same inputs unless the team spirit hyperparameter is exactly 1, which it is not. You are partially correct in that the agents consume the parameters of the four other agents, but it is weighted differently accorsing to team spirit parameters.
      - backpropaganda 8y ago
        The team spirit hyperparameter is a crutch they've introduced themselves. Ideally it should be one. In Dota there is only one objective for the entire team and that's to win the game. The fact that they shape rewards is an implementation detail and doesn't change the fact that Dota 2 does not require cooperation, because there's no cooperation game being played. It's a purely zero-sum adversarial game being played between two teams.
        deleted 8y ago
        [deleted]
gsich 8y ago
"very similar" in terms of getting regeneration all the time. 5 couriers means deathpush. Simple mistakes from the humans scale up, games won't last long (around 20 minutes), so no long-term strategy is needed. It's not fun to watch some mechanical skill outplaying humans. You can compare it to aimbots in ego-shooters. Yes they aim better, but that's it. I think they should have trained with 1 courier right from the beginning.
- duskwuff 8y ago
  5 couriers also meant hyper-aggressive behavior in the laning stage, because regen is so easy to come by. If I had to guess, they started out with the 5-courier setup because otherwise the bots would end up fighting over the courier -- just like a typical pub team. :)
  - backpropaganda 8y ago
    There are many things you could do to make the bots not fight for courier. Just implement a queuing system with a priority for cores. Or better yet, make the couriers not usable if another hero is using it. The way I would guess this played out was that OpenAI tried multiple variants of the game, and this version ended up being good enough to beat human players. Of course, they later learnt that the bots were winning not because they were good, but the variant was imbalanced and its strategies unfamiliar to humans.
    - gdb 8y ago
      > The way I would guess this played out was that OpenAI tried multiple variants of the game, and this version ended up being good enough to beat human players No. We had implemented scripted courier logic for 1v1, and when switching to 5v5 the easiest starting point was to run five copies of that logic. (Dota's Turbo mode also has five invulnerable couriers.) In June, we had more important restrictions to remove — such as wards & Roshan; you personally were focused on the particular heroes we'd chosen: https://news.ycombinator.com/item?id=17392455 https://news.ycombinator.com/item?id=17392455. The couriers only become our most important restriction in August.
kenhwang 8y ago
Maybe from an AI perspective, it's impressive that the bots almost managed to learn the rules of the (still drastically simplified subset) game. From a game skill point of view, it's a joke; it's incredibly obvious that the bots were only able to prolong the game due to consistently better than human mechanical skill. Of the list of skills you mentioned that the bots "learned", on the main stage, the draft was provided by humans, and from a significantly limited set, the bots never pressured highground, checked for Roshan when it was impossible for it to be up, had nonsensical placement of vision (placed vision where vision already existed), continuously invested in anti-invisibility consumables even when the opposition team had no invisibility. If the mechanical skills were toned down to human levels, I don't think the latest iteration of the bot on the latest ruleset can even compete with the average human player, much less the pros.

alper111 8y ago

If you look at network structure, it acts as one agent, not five. So, free coordination. (See: https://t.co/GPKHPsIu1C https://t.co/GPKHPsIu1C) In my opinion, what i see is a very good player who knows how to chain stun precisely without any strategic depth. If you claim to have built an AI system, which you ultimately want it to evolve to AGI, you at least expect some sort of strategic decision making at the macro level. Though since it has almost perfect micro, it can easily outweight the most of teams. So yeah, with that expectation I see this as a joke, too. P.S. The model is trained with 128k cpus and 256 gpu. It is able to play 180 years worth of game in a day. Think about it.

whymauri 8y ago
It's five independent agents. The article on OpenAI's website and the network structure both say this. I'll zoom in since it's a complicated structure. It's the first line of the article: https://blog.openai.com/openai-five/ https://blog.openai.com/openai-five/ >Our team of five neural networks, They use a hyperparameter called team spirit to cooperate. I don't think the goal of this is AGI at all, so I don't see why people are making that leap. But sure, for the geniuses of HN this must clearly be trivial.
- backpropaganda 8y ago
  It's not independent agents. The neural networks have the same input, share weights, and also share some activations. With that much sharing, it's better to think of it as one neural network which has output heads for all the 5 players. So coordination is free. Actually, coordination makes as little sense as saying that multiple neurons in a neural network are cooperating, or that the two legs of a humanoid are cooperating to walk. Further there is no game being played between heroes of the same team. They literally have the same objective. The "coordination" buzzword is just another attempt by OpenAI to confuse and mislead readers, and give a false sense of their progress.
  - whymauri 8y ago
    They cannot share the same inputs unless the team spirit hyperparameter is exactly 1, which it is not. You are partially correct in that the agents consume the parameters of the four other agents, but it is weighted differently accorsing to team spirit parameters.
    - backpropaganda 8y ago
      The team spirit hyperparameter is a crutch they've introduced themselves. Ideally it should be one. In Dota there is only one objective for the entire team and that's to win the game. The fact that they shape rewards is an implementation detail and doesn't change the fact that Dota 2 does not require cooperation, because there's no cooperation game being played. It's a purely zero-sum adversarial game being played between two teams.
      - deleted 8y ago
        [deleted]

gsich 8y ago

"very similar" in terms of getting regeneration all the time. 5 couriers means deathpush. Simple mistakes from the humans scale up, games won't last long (around 20 minutes), so no long-term strategy is needed. It's not fun to watch some mechanical skill outplaying humans. You can compare it to aimbots in ego-shooters. Yes they aim better, but that's it. I think they should have trained with 1 courier right from the beginning.

duskwuff 8y ago
5 couriers also meant hyper-aggressive behavior in the laning stage, because regen is so easy to come by. If I had to guess, they started out with the 5-courier setup because otherwise the bots would end up fighting over the courier -- just like a typical pub team. :)
- backpropaganda 8y ago
  There are many things you could do to make the bots not fight for courier. Just implement a queuing system with a priority for cores. Or better yet, make the couriers not usable if another hero is using it. The way I would guess this played out was that OpenAI tried multiple variants of the game, and this version ended up being good enough to beat human players. Of course, they later learnt that the bots were winning not because they were good, but the variant was imbalanced and its strategies unfamiliar to humans.
  - gdb 8y ago
    > The way I would guess this played out was that OpenAI tried multiple variants of the game, and this version ended up being good enough to beat human players No. We had implemented scripted courier logic for 1v1, and when switching to 5v5 the easiest starting point was to run five copies of that logic. (Dota's Turbo mode also has five invulnerable couriers.) In June, we had more important restrictions to remove — such as wards & Roshan; you personally were focused on the particular heroes we'd chosen: https://news.ycombinator.com/item?id=17392455 https://news.ycombinator.com/item?id=17392455. The couriers only become our most important restriction in August.

kenhwang 8y ago

Maybe from an AI perspective, it's impressive that the bots almost managed to learn the rules of the (still drastically simplified subset) game. From a game skill point of view, it's a joke; it's incredibly obvious that the bots were only able to prolong the game due to consistently better than human mechanical skill. Of the list of skills you mentioned that the bots "learned", on the main stage, the draft was provided by humans, and from a significantly limited set, the bots never pressured highground, checked for Roshan when it was impossible for it to be up, had nonsensical placement of vision (placed vision where vision already existed), continuously invested in anti-invisibility consumables even when the opposition team had no invisibility. If the mechanical skills were toned down to human levels, I don't think the latest iteration of the bot on the latest ruleset can even compete with the average human player, much less the pros.