Edit: The parent either edited his comment or I replied to the wrong one. He was suggesting to use a second agent to detect if the player is cheating. Use it t

Edit: The parent either edited his comment or I replied to the wrong one. He was suggesting to use a second agent to detect if the player is cheating. Use it to correct the first LLM when it produces bad replies (allowing the player to cheat, handling anachronic elements informatively, etc). Build up a dataset. Fine-tune. In short, it's less of a reasoning problem than a matter of misalignment of the LLM's personality/role. I'm using the word "alignment" here because I believe the kind of behavior people have noted in this comment thread is the result of what "AI alignment" has come to mean. A helpful assistant makes for a bad dungeon master. On a tangent line I think it's also one of the main component that make us wish LLM were more "agentic". When was the last time a LLM asked you to put more info in its context ? Imagine you're using an LLM to assist you in implementing something in a vast code base. Have you ever had a LLM asking you to provide the missing .cpp corresponding to a .h you have fed it ? Has a LLM ever asked you to run a python script and copy-paste the result into its context so that it can have access to a map of the repo you're working on ? LLMs aren't proactive enough and in light of what was reported before they were aligned, I tend to think it is a "feature", not a bug. Don't forget there was a time when GPT4 would reach out to people on TaskRabbit to have them solve a captcha. >We granted the Alignment Research Center (ARC) early access to the models as a part of our expert red teaming efforts in order to enable their team to assess risks from power-seeking behavior. The specific form of power-seeking that ARC assessed was the ability for the model to autonomously replicate and acquire resources >[...] Preliminary assessments of GPT-4’s abilities, conducted with no task-specific finetuning, found it ineffective at autonomously replicating, acquiring resources, and avoiding being shut down “in the wild.” Source: https://cdn.openai.com/papers/gpt-4.pdf https://cdn.openai.com/papers/gpt-4.pdf