While I grant that it doesn't have the ability to go into single-file or line-level granularity, GPT-4 can have a highly informed discussion on the architecture

While I grant that it doesn't have the ability to go into single-file or line-level granularity, GPT-4 can have a highly informed discussion on the architectures of big projects in its training set like PostgreSQL or the Linux kernel. That's what I mean by "holistic understanding". If you're a contributor for one of those projects (or want to be), it could definitely help to evaluate the tradeoffs of various design approaches for new features or modifications. Getting into the nitty gritty of files and functions would be a step further, but it seems to me that just having this general knowledge of how a big project is structured would vastly improve any output related to that project compared to just loading in comparatively tiny pieces of it. Even if those tiny pieces are the most relevant to the prompt, it will be missing all the background context that it has for projects in its training set.