The issue is that LLMs' handling of "speak like a pirate" is evaluated by humans with a preconceived idea of what coherently expressed pirate-style responses sh

The issue is that LLMs' handling of "speak like a pirate" is evaluated by humans with a preconceived idea of what coherently expressed pirate-style responses should look like (and their outputs significantly improved by human feedback loops). We also have linters, compilers and interpreters to evaluate correctness of code and reject stuff that's borrowed syntax from the wrong language. I can't see any obstacle to an LLM ingesting a database of squawks and emitting squawks which may or may not be in relevant patterns consistent with how a particular flock of birds communicate, but I can't see the birds advising us on whether it's using them idiomatically or not.