I tried Google's on-device llm recently and the question I asked it was "how common are [partner's fairly rare surname]s?".^ It answered by detailing a scenari

I tried Google's on-device llm recently and the question I asked it was "how common are [partner's fairly rare surname]s?".^ It answered by detailing a scenario where they are a secret society etc. I'm amazed the main providers have managed to get the large ones to stay on the rails as much as they do. They were trained on the internet after all ^I know. The reply about how they're next token predictors, I know.