3 ms·
We've all tried to ask the LLM about something outside of its training data by now. In that situation, they give the (wrong) answer that sounds the most plausi
by andy81 7mo ago
We've all tried to ask the LLM about something outside of its training data by now.
In that situation, they give the (wrong) answer that sounds the most plausible.
- PacificSpecific 7mo agoThat's definitely been my experience. I work with a lot of weird code bases that have never been public facing and AI has horrible responses for that stuff. As soon as I tried to make a todomvc it started working great but I wonder how much value that really brings to the table. It's great for me though. I can finally make a todomvc tailored to my specific needs.
- ctoth 7mo agoI'm not sure what sorts of weird codebases you're working with but I recently saw Claude programming well on a Lambda MOO -- weirder than that?
- PacificSpecific 7mo agoI had to Google that haha. It's in that realm but more complex. I do plan to repeatedly come back and try though. Just so far it hasn't been useful.
- NoMoreNicksLeft 7mo agoOnce or twice, for me it's deflected rather than answer at all. On the other hand, they've also surfaced information (later independently confirmed by myself) that I had not been able to find for years. I don't know what to make of it.
- visarga 7mo ago> In that situation, they give the (wrong) answer that sounds the most plausible. Not if you use web search or deep report, you should not use LLMs as knowledge bases, they are language models - they learn language not information, and are just models not replicas of the training set.