5 ms·
That’s not even a very large code base. My experience is definitely that anything with more than 100K-loc really makes the LLMs struggle.
by ch4s3 1y ago
That’s not even a very large code base. My experience is definitely that anything with more than 100K-loc really makes the LLMs struggle.
- serf 1y agothere is more to it than that. it's about modularization as well. I run LLMs against a 500k LoC poker engine and they do well because the engine is modularized into many small parts with a focus on good naming schemes and DRY. If it doesn't require a lot of context for an LLM to figure out how to direct effort then the codebase size is irrelevant -- what becomes relevant in those scenarios is module size and the amount of modules implicated with any change or problem-solving. The LLM codebase 'navigation' becomes near-free with good naming and structure. If you code in a style that allows an LLM to navigate the codebase via just an `ls` output it can handle things deftly. The LLMification of things has definitely made me embrace the concept of program-as-plugin-loader more-so than ever before.
- IanCal 1y agoThis has the side benefit of likely being easier to navigate for humans too. The less I need to keep in my head to figure something out the better.
- ch4s3 1y agoThe app I work on is fairly highly modular, to the point that we split the app in half and unwinding the two halves of the code only took about 2 weeks. > The LLM codebase 'navigation' becomes near-free with good naming and structure I have not found this to be true. They seem to break badly if you have a lot of files with similar-ish names even if they're descriptive.