3 ms·
I wonder why Claude’s answers aren’t equal or better than Wikipedia - assuming Wikipedia is one of the training datasets. Is the temperature causing it to be pr
by the_arun 1y ago
I wonder why Claude’s answers aren’t equal or better than Wikipedia - assuming Wikipedia is one of the training datasets. Is the temperature causing it to be probabilistic & other sources are carrying more weight?
- lazide 1y agoYou can think of a LLM as a type of lossy compression of knowledge. With that in mind, is it really surprising that you don’t get the ‘right’ answer out? Any more than if you compress an image with JPEG, a given pixel isn’t the ‘right’ color anymore either? They’re both close (kinda) at least, which is the point. If you wanted the exact right answer, don’t use lossy compression - it’ll be expensive in other ways though.
- ksenzee 1y agoWhat a great metaphor. I’m adopting that immediately, thank you.