OpenAI's text completion has an option to return "log probability" or something for each token. That might apply. You can also turn down the temperature paramet

OpenAI's text completion has an option to return "log probability" or something for each token. That might apply. You can also turn down the temperature parameter which reduces hallucinations to some degree.