4 ms·

I’m very far from an evangelist of the tech but I was under the impression they had gotten better at this sort of thing. I question why a model which “knows” a

I’m very far from an evangelist of the tech but I was under the impression they had gotten better at this sort of thing. I question why a model which “knows” about cheddar / mozzarella cheese would make this blunder. Was this supposed to be generated by one of those “show your work” reasoning models or is this just the regurgitation of one of the single short response parrot of old quora answer or reddit post ai chatbots?

alt227 2y ago
My understanding off LLMs is not great, but Im pretty sure that when the glorified predictive text come up with a sentance, it doesnt cross reference that with any other knowledge to verify it. It just spits it out.
krackers 2y ago
Maybe it's because of the way the search integration is set up? All the search results are put in the context window and the model is just told to summarize it, so that's what it dutifully does, without considering whether or not the statements are plausible. Even without one of the reasoning models, if the prompt were adjusted to tell it that some of the statements may be incorrect and that information should be evaluated for plausibility first, then the result would probably be better. I asked 4o and Sonnet to "Evaluate the plausibility of this statement: Gouda is the most popular Dutch cheese in the world, accounting for 50 to 60% of the world's cheese consumption". 4o correctly said >While Gouda is an iconic and highly popular Dutch cheese, the claim that it accounts for 50–60% of the world's cheese consumption is implausible and likely a misrepresentation of its market share. Global cheese consumption is far too diverse for any single variety, including Gouda, to dominate to such an extent." >Sonnet says "A more accurate statement would be that Gouda is one of the world's most popular cheeses and represents a significant portion of Dutch cheese production and exports. It's estimated that Gouda accounts for over 60% of Dutch cheese production, which might be where the confusion stems from." Both correctly pointed out mozzarella, cheddar, and paremesan as the actual likely candidates for most popular cheese. So the models are clearly quite capable of this, any bad result is likely just prompting error, e.g. blindly asking for a summary.

alt227 2y ago

My understanding off LLMs is not great, but Im pretty sure that when the glorified predictive text come up with a sentance, it doesnt cross reference that with any other knowledge to verify it. It just spits it out.

krackers 2y ago

Maybe it's because of the way the search integration is set up? All the search results are put in the context window and the model is just told to summarize it, so that's what it dutifully does, without considering whether or not the statements are plausible. Even without one of the reasoning models, if the prompt were adjusted to tell it that some of the statements may be incorrect and that information should be evaluated for plausibility first, then the result would probably be better. I asked 4o and Sonnet to "Evaluate the plausibility of this statement: Gouda is the most popular Dutch cheese in the world, accounting for 50 to 60% of the world's cheese consumption". 4o correctly said >While Gouda is an iconic and highly popular Dutch cheese, the claim that it accounts for 50–60% of the world's cheese consumption is implausible and likely a misrepresentation of its market share. Global cheese consumption is far too diverse for any single variety, including Gouda, to dominate to such an extent." >Sonnet says "A more accurate statement would be that Gouda is one of the world's most popular cheeses and represents a significant portion of Dutch cheese production and exports. It's estimated that Gouda accounts for over 60% of Dutch cheese production, which might be where the confusion stems from." Both correctly pointed out mozzarella, cheddar, and paremesan as the actual likely candidates for most popular cheese. So the models are clearly quite capable of this, any bad result is likely just prompting error, e.g. blindly asking for a summary.