3 ms·
Show HN: I had models play Slime Soccer against each other
There were a few things that I wanted from this ai benchmark:
* The models to compete directly with each other
* To be able to _see_ how they were competing
* Something more fun than a math test
Slime Soccer (a web game from my childhood) was like a good option since the decision set is small for each move and it felt nostalgic.
The most surprising thing is that the flagship models from each lab didn't always do best. For example, GPT 5.6 Sol was almost the worst performing model from OpenAI.
- zecice 3mo agolooks cool! how did you decide when to ask each model for a move? or did you ask them to write an algorithm to play?
- cshdllr 3mo agoThanks! About three times a second each model gets a fixed system prompt plus the current game state as JSON, and replies with one action word (move_right, move_left, jump, idle). This is all run as a simulation since they couldn't reply fast enough for it to be realtime. Then the video is made after the fact from the logs.
- gargarbot 3mo agoWhy do you think Sol performed so badly compared to other frontier models?
- cshdllr 3mo agoI honestly don't know! My guess is that Sol is better calibrated for the hard questions that require more reasoning...but even still I'd expect it to be able to handle a simple game like this better than its sibling models.
- tylerjaywood 3mo agoNeat! I'd love to be able to play against a model. Do you know how they fare against human slime soccer players?
- cshdllr 3mo agoSadly I think most human Slime Soccer players have retired. But more seriously, the way this is hooked up is not realtime, so the game would require pauses while the model decided what to do. Maybe next I can try more of a turn based RPG which would more easily incorporate human players.