> so there was basically no chance that models had been trained on submission code yet. I do suspect performance on AoC 24 will increase over time… No, but sev

> so there was basically no chance that models had been trained on submission code yet. I do suspect performance on AoC 24 will increase over time… No, but several of the problems were similar to problems in the training set from other challenges. There was one that was almost exactly the same as one on Google Foobar called 'Prepare the Bunnies' Escape' as an example. It is pretty hard to make these challenges that work across multiple langs, skills etc.. without making them into 'which works dfs/bfs/a*/dykstra' problems. Obviously outside of the leaderboard group, an LLM being able to solve it doesn't really matter. My friends and I flipped a coin for TDD vs traditional with each day, testing out assumptions and evaluating what was most readable/maintainable. It was great for that.