4 ms·
Yes, there’s research showing that models’ self-assessment of probabilities (when you ask them via prompting) don’t even match the same models’ actual probabili
by _delirium 2y ago
Yes, there’s research showing that models’ self-assessment of probabilities (when you ask them via prompting) don’t even match the same models’ actual probabilities, in cases where you can measure the probabilities directly (e.g. by looking at the logits): https://arxiv.org/abs/2305.13264 https://arxiv.org/abs/2305.13264
- anon291 2y agoLogits are not probabilities... at least not in the way you understand probability. Probabilities mathematically are anything that broadly behaves like a probability, whereas colloquially probabilities represent the likelihood or the preponderance of a particular phenomenon. Logits are not either of those.