While this is a neat demo, I would in general be cautious about translating GPT-4's logits to probabilities. We know from the technical report that its confiden

While this is a neat demo, I would in general be cautious about translating GPT-4's logits to probabilities. We know from the technical report that its confidences are not well calibrated. See Figure 8: https://arxiv.org/pdf/2303.08774.pdf https://arxiv.org/pdf/2303.08774.pdf