> It’s a very common observation that the farther you stray from mainstream, the less effective the LLM models become. It is a common observation but I don't b

> It’s a very common observation that the farther you stray from mainstream, the less effective the LLM models become. It is a common observation but I don't buy it. AI is clearly very good at Rust, but that is probably one of the least represented languages in its dataset. Anecdotally, I've also been having very good outcomes with a rather niche combination of technologies (opencv.js + JS in a browser extension) since early 2024. I would imagine there is way more C++ game code in the training set than that particular combination. I think the more likely reason is that certain languages, projects or technologies tend to be organized in ways that are not ideal for LLMs. Specifically, I think Object Oriented approaches are not ideal for LLMs. My theory is the key factor for effective LLM use is how effectively you can stuff the context with only the relevant data. OO tends to result in logic spread across inheritance hierarchies and templates (and even overloaded operators /shudder) which resides in a bunch of different files comingled with a whole lot of other logic. This just tends to confuse the LLM. On the other hand, I ended up using a lot more functional programming style which let me pinpoint the exact files or snippets of code relevant to a task, and the LLM pretty much never went wrong. These days the models (and likely the harnesses) are much stronger and need much less curation of context, and hence can power through any kind of project organization. But I suspect they are still a bit sensitive to all the noise polluting their contexts and hence can produce very inconsistent results.