3 ms·
Kradle: Eval AI with Simulations
- alexisgauba 1y agovery cool. how are ya'll thinking about genie 3?
- jamest 1y agoGenie 3 is super impressive -- but you can't manipulate it programmatically like Minecraft. We've built our infra so that we can plug in any simulation environment. If an AI-generated world starts being programmatically modifiable (and has really solid object permanance :) then we'd happily use it!
- stopachka 1y agoInteresting work! This can be a great way to have ML models live in a sandbox and prove out some of the safety concerns Anthropic et al speak about.
- jamest 1y agoHi HN - Minecraft has a DSL that lets you manipulate the world. We've piggy-backed on that, along with a K8s infra to run N worlds in parallel, to let you create simulations of arbitrary complexity. We think simulations are the best way to test frontier AIs due to their degrees of freedom and expressivity. AMA!
- benhylak 1y agohow do you see the world-simulation approach overlapping with the sort of custom-RL environments (e.g. mock Salesforce apps) that people are building out for frontier labs? similarly, do you see it as a general test of intelligence? more for robotics?