I'm pretty excited about the increased context length (e.g. in my other comment here[0]), but I'm kind of disappointed by the examples here. The codebase is 10

I'm pretty excited about the increased context length (e.g. in my other comment here[0]), but I'm kind of disappointed by the examples here. The codebase is 100k lines, but the tasks it gave it seemed to be focused on just hundreds of examples. The examples are probably largely independent, so it doesn't seem like this is really flexing anything a relatively simple RAG approach with a much smaller context window couldn't handle. The prompts said "the demo that ...", so it's a matter of identifying the demo in question and looking at just that code, which is a much smaller necessary context. There was the "use the GUI approach from other examples" task, which kind of gets there, but that's kind of another distinct little bit of code. In other words, while the codebase has lots of lines, the actual inference across them seemed to use relatively few of them, and identifying the relevant lines didn't seem that hard to me based on the tasks given. That means it could be done with some retrieval and a much smaller context window. From the title, I thought it would be loading 100k into the context and then asking some deeper questions like "find the bug" that spans several function calls or something like that. Something that wouldn't be trivial to accomplish with current techniques. [0] https://news.ycombinator.com/context?id=39384034 https://news.ycombinator.com/context?id=39384034