You could use a similar engine to keep track of things and feed them to GPT could provide new descriptions or ideas, or use functions provided by the engine to

You could use a similar engine to keep track of things and feed them to GPT could provide new descriptions or ideas, or use functions provided by the engine to modify map state or run appropriate commands according to the what the user says. To start, it could be as simple as giving GPT the list of valid commands for the user state and location and having it translate if the user asks for something that is equivalent but doesn't match the command exactly.