2 ms·
I think the lesson is: smaller models hallucinate more, so only use them in your applications where you load up large prompts with specific data to reason about
by mark_l_watson 1y ago
I think the lesson is: smaller models hallucinate more, so only use them in your applications where you load up large prompts with specific data to reason about. Then even the small Google gemma3n 4B model can be amazingly useful.
I use the SOTA models from Google and OpenAI mostly for getting feedback on ideas, helping me think through designs, and sometimes for coding.
Your question is clearly best answered using a large commercial model with a web search tool. That said, integrating a local model with a home built interface to something like the Brave search API can be effective but I no longer make the effort.
- 9rx 1y ago> think the lesson is: smaller models hallucinate more The interesting part isn't the hallucination, but the sheer unwillingness to take in new information.
- dragonwriter 1y agoMight have dug in less on the confabulation about information outside of its knowledge cutoff if the new information weren't offered with support from a user “hallucination” about information within its knowledge cutoff. More detail: https://news.ycombinator.com/item?id=44809145 https://news.ycombinator.com/item?id=44809145
- 9rx 1y ago> Might have dug in less... The digging in at all is what is interesting. Like an earlier comment alluded to, the presumptive value of these tools is being able to feed it your own information where that information is to be considered authoritative. > More detail: [...] "He, in fact, did run for reelection" A slow walk, maybe. He was in no condition to run. That is why he ultimately dropped out. But, really, that statement is just a silly game of semantics. "Run", when used in hindsight, often implies completion. This LLM model even says so too. If a model doesn't understand nuance, that is also interesting.