I would suggest that example is more likely evidence of semantic associations provided by underlying taxonomy based on input training data. There is so much con

I would suggest that example is more likely evidence of semantic associations provided by underlying taxonomy based on input training data. There is so much contextual information in language hierarchy and associated characteristics and there is nothing stopping a LLM from using multiple recombinant factors for generation. I don’t mean to discount your excellent comment. I mean obviously these models are so complex already that their black box nature makes it difficult to derive conclusive results, but Arkham’s razor would imply that the simplest explanation — that is it recombined training data creating a heirarchy with multiple factors that provides context for the “correct” answer — that is more likely correct. In other words a LlM can confer multiple related associations to output based on training data, and that is the most likely explanation for that behavior. Just look at the poor performance with GPT-3 and the underlying data quality issues the Alpaca dataset team has discussed. I’m no expert, and don’t mean to counter you as much as contribute my own limited thinking for discussion here. Cheers!