5 ms·

I’ve been dying for someone to make a Civilization AI. It might not be too crazy of an idea - would love to see a model fine-tuned on sequences of moves. The

I’ve been dying for someone to make a Civilization AI. It might not be too crazy of an idea - would love to see a model fine-tuned on sequences of moves. The biggest limitation of video game AI currently is not theory, but hardware. Once home compute doubles a few more times, we’ll all be running GPT-4 locally and a competent Civilization AI starts to look realistic.

Buttons840 3y ago
I think it's also a matter of "shape". Like, GPT4 solves one "shape" of problem, given tokens, predict the next token. That's all it does, that's the only problem it has to solve. A Civilization AI would have many problem "shapes". What do I research? Where do I build my city, what buildings do I build, how do I move my units, what units do I build, what improvements do I build, when do I declare war, what trade deals do I accept, etc, etc. Each of those is fundamentally different, and you can maybe come up with a scheme to make them all into the same "shape", but then that ends up being harder to train. I would be interested to see a good solution to this problem.
- dannyw 3y ago
  You can constrain LLMs (like LLAMA) to only output certain tokens that match some schema (e.g. valid code syntax). I don't see why you can't get a LLM to output something like "research tech332; build city3 building24".
Baeocystin 3y ago
I am 100% certain that the training of such an AI will result in winning a game without ever building a single city* and 1,000 other exploits before being nerfbatted enough to play a 'real' game. (That doesn't mean I don't want to see the ridiculousness it comes up with!) * https://www.youtube.com/watch?v=6CZEEvZqJC0 https://www.youtube.com/watch?v=6CZEEvZqJC0
- jdietrich 3y ago
  If you train the model purely based on win rate, sure. Fortunately, we can efficiently use RLHF to train a model to play in a human-like way and give entertaining matches.
- lordnacho 3y ago
  I knew it, I knew it! It would be a Spiffing Brit video. That guy is a genius at finding exploits in computer games. I don't know how he does it, I think you need to play a fair bit of each game before you find these little corners of the ruleset.
  - boppo1 3y ago
    Idk maybe he uses some sort of fuzzer
- Aerroon 3y ago
  But wouldn't this be amazing for the developer to fix a lot of edge cases/bugs?
  - Baeocystin 3y ago
    Maybe, maybe not. The stochastic, black-box nature of the current wave of ML systems gives me a gut feeling that using them like this is more of a Monkey's Paw wish granter than useful tool without a lot of refinement first. Time will tell!
amelius 3y ago
> I’ve been dying for someone to make a Civilization AI. Would love to see someone to make an AI that can predict our economy, perhaps by modeling all the actors that participate in the economy using AI agents.

Buttons840 3y ago

I think it's also a matter of "shape". Like, GPT4 solves one "shape" of problem, given tokens, predict the next token. That's all it does, that's the only problem it has to solve. A Civilization AI would have many problem "shapes". What do I research? Where do I build my city, what buildings do I build, how do I move my units, what units do I build, what improvements do I build, when do I declare war, what trade deals do I accept, etc, etc. Each of those is fundamentally different, and you can maybe come up with a scheme to make them all into the same "shape", but then that ends up being harder to train. I would be interested to see a good solution to this problem.

dannyw 3y ago
You can constrain LLMs (like LLAMA) to only output certain tokens that match some schema (e.g. valid code syntax). I don't see why you can't get a LLM to output something like "research tech332; build city3 building24".

Baeocystin 3y ago

I am 100% certain that the training of such an AI will result in winning a game without ever building a single city* and 1,000 other exploits before being nerfbatted enough to play a 'real' game. (That doesn't mean I don't want to see the ridiculousness it comes up with!) * https://www.youtube.com/watch?v=6CZEEvZqJC0 https://www.youtube.com/watch?v=6CZEEvZqJC0

jdietrich 3y ago
If you train the model purely based on win rate, sure. Fortunately, we can efficiently use RLHF to train a model to play in a human-like way and give entertaining matches.
lordnacho 3y ago
I knew it, I knew it! It would be a Spiffing Brit video. That guy is a genius at finding exploits in computer games. I don't know how he does it, I think you need to play a fair bit of each game before you find these little corners of the ruleset.
- boppo1 3y ago
  Idk maybe he uses some sort of fuzzer
Aerroon 3y ago
But wouldn't this be amazing for the developer to fix a lot of edge cases/bugs?
- Baeocystin 3y ago
  Maybe, maybe not. The stochastic, black-box nature of the current wave of ML systems gives me a gut feeling that using them like this is more of a Monkey's Paw wish granter than useful tool without a lot of refinement first. Time will tell!

amelius 3y ago

> I’ve been dying for someone to make a Civilization AI. Would love to see someone to make an AI that can predict our economy, perhaps by modeling all the actors that participate in the economy using AI agents.