Expectations vary wildly. Sometimes people expect to use LLMs to unearth hard to find information. In reality, LLMs seem to quickly fall apart when you go fro

Expectations vary wildly. Sometimes people expect to use LLMs to unearth hard to find information. In reality, LLMs seem to quickly fall apart when you go from ubiquitous libraries with 200k stars on GitHub to one with "just" 1k stars. What makes the situation worse is the way LLMs fail. Hallucinations where it goes "my usage example did not work because you are on the wrong version of the library/using the wrong SDK" etc. are super common in this scenario. This leads to further time wasted trying to apply reasonably plausible fixes that are entirely hallucinated.