You can think of a LLM as a type of lossy compression of knowledge. With that in mind, is it really surprising that you don’t get the ‘right’ answer out? Any m

You can think of a LLM as a type of lossy compression of knowledge. With that in mind, is it really surprising that you don’t get the ‘right’ answer out? Any more than if you compress an image with JPEG, a given pixel isn’t the ‘right’ color anymore either? They’re both close (kinda) at least, which is the point. If you wanted the exact right answer, don’t use lossy compression - it’ll be expensive in other ways though.