4 ms·
I think this one has advantages over the “pelican riding a bicycle” one because it hinges on an anatomical feature that many models associate with royalty, “hab
by thebigship 2mo ago
I think this one has advantages over the “pelican riding a bicycle” one because it hinges on an anatomical feature that many models associate with royalty, “habsburg” being a lineage and “habsburg jaw” being an anatomical feature.
Seven of fourteen models silently imported royalty into a prompt that named only an anatomical feature. Two of them knew they were extrapolating ("because Habsburg") and did it anyway.
Mistral returned byte-identical output across separate calls.
Gemini narrates its work in 65 comments; Llama says nothing.
If you're deciding which model to trust with instructions, "how much does it embellish beyond what I asked" and "does it behave deterministically" are directly practical questions.
- HPsquared 2mo agoBite-identical?
- deleted 2mo ago[deleted]
- thebigship 2mo agoclearly I missed an amazing copy opportunity, thank you haha
- n00bskoolbus 2mo agoThe identical pair from Mistral took me off guard. Many of the other models were so varied between the runs which is more what I would expect.
- fwip 2mo agoI wonder if the setup accidentally hit a cache at some layer.
- zahlman 2mo ago> Two of them knew they were extrapolating ("because Habsburg") and did it anyway. You seem to imply that they ought not to. I disagree. I wasn't familiar with the term before this post. Having learned it, were I given the task, I think I'd be strongly tempted to do the same extrapolation. > If you're deciding which model to trust with instructions, "how much does it embellish beyond what I asked" and "does it behave deterministically" are directly practical questions. Agency is agency. You still need to vet what the model's output is actually permitted to control.
- thebigship 2mo ago> You seem to imply that they ought not to. I disagree. That would be like saying anyone with Lou Gehrig's disease must look like Lou Gehrig. So we'll have to agree to disagree here.
- zahlman 2mo agoWell, no, I think you create a pretty clear false dichotomy there. Certainly a person with Lou Gehrig's disease could look like Lou Gehrig; and especially in a cartoon illustration, where it's difficult to convey the point, this kind of artistic license is used specifically so that the viewer will make these kinds of associations. I would, likely, otherwise perceive an underbite as just an underbite.