3 ms·
Codex on GPT6 Astra Low launched a rocket into space in Factorio Space Age 2.1
- Echo4309 25d agoThat’s pretty impressive. I wonder how they handled biter attacks and some of the real-time requirements.
- nfemmanuel 19d ago[dead]
- dude250711 25d agoGetting good coding results with Astra Ultra agent and Astra Low subagents.
- iJohnDoe 25d agoWhat does Ultra offer in this context? Better code? Or is the problem being solved extremely complicated that requires Ultra?
- TuxSH 25d agoUltra is max reasoning level that splits the work and spawn subagents. Good for whole-project reviews (well, as long as the cybersecurity refusal doesn't appear for whatever reason), but unusable on the $20/mo tier
- epaga 25d agoAs far as I understand, the best any AI was able to get to until now was being able to produce blue science which is like level 2 or 3 of a science tree like 10 levels deep. To me who has maybe put 20-30 hours into Factorio, this is even more impressive stuff than some of the Blender videos I've seen... just mind-boggling.
- psihius 24d agoSince I posted, it got to Fulgora and Vulcanus, successfully colonized then and at the moment is preparing to tackle Gleba. It's not even showing signs of stopping or getting stuck.
- Starman_Jones 25d agoLet’s see the factory layouts though. Release the blueprints!
- __s 25d agoThere's vod: https://m.twitch.tv/videos/2867906622 https://m.twitch.tv/videos/2867906622
- parasxos 25d agoThe factory must grow, and now it can grow while you sleep, which is the one thing Factorio players never managed.
- loa_in_ 25d agoSleep is a bottleneck
- ramblerman 25d agoCan someone ELI5 how LLMs are manipulating environments like this? I have a similar question for when you see the blender examples online. AFAIU the models are multimodal, so you could screenshot factorio and let it come back with some manipulation like move west 5 tiles, do that and then take another screenshot, but that seems wildly inefficient so I'm sure i'm missing something
- indoorcomic 25d agoThere are people much smarter than me to answer this, but the basic idea are that models have a harness around them that include something called "tools." Each tool is basically a text description of its functionality. So for something like this, for example, the model might have a "move" tool. All of the tools are added to the model's context. Then the basic idea is that when the model detects it needs to "move west 5 tiles" it will spit out a bunch of JSON to "call" the tool, and then at that point a normal program picks up that tool call output and actually performs the action. So in other words, the LLM isn't directly manipulating the environment, it's spitting out JSON that tells the harness it is in what to do.
- embedding-shape 25d agoSimplest general way is literally asking the model for the X and Y coordinates to click, given an input image. Tends to work with most multi-modal models, although some have better accuracy than others, obviously. More involved integration, would be to expose reading and writing to the game state or character controls via tools that the LLM can use (either in your own harness however you want, or via MCP). This is basically what most "AI tooling" connected to editors do today, one way or another, with fallback to "click on this UI element after taking a screenshot" often, or "simulate click on this widget with #id". So given a goal, the LLM would first call check_inventory(), then craft_building(:building), then walk_to_coords(x,y), then build_building(:building) and so on, which tends to indeed be a lot faster than doing it by screenshots and inferring coordinates. Ultimately I think the harness and prompt matters more than the model here, but I guess what they're exploring, is the shortest possible prompt to have it do it correctly. Otherwise, setting your harness to "Continue until you've launched a rocket in the game" together with a bunch of added rules to the prompts as you discover where it gets stuck, is an easy way to probably get most LLMs to be able to do this correctly and relatively fast. Some of them requires more handholding than others, which I guess might be another thing they're testing for here, it isn't 100% clear what the intention/motivation nor exact environment was/is. Edit: As fitting as it is, I was just trying out the Unreal Engine Editor MCP/tools for the first time with Codex, which ended up also being a good example for how a "real turn" would look like, with a LLM connected to some other editor: https://gist.github.com/embedding-shapes/b16c50ed591569d3c7013f60206be4bf https://gist.github.com/embedding-shapes/b16c50ed591569d3c70...
- majora2007 25d agoHow is the AI controlling the video game? Is there special coding/harness that allows for this?
- psihius 24d agoFactorio has an extremely broad LUA modding API, so a mod was made that allows it to get data about the game and send input actions to control the game. It also has ability to make screenshots and it does use it from time to time to inspect whats going on.
- mrgoldenbrown 25d agoLots of people in here asking how it was done. Heres what the linked page says: >...made a mode in LUA for the game to expose an MCP server that allows the llm to play the game. (Factorio has a very mature LUA based modding system )