> “GPT-3 is not necessarily well-calibrated in its predictions on novel inputs.” I don't thing even humans are well calibrated on novel inputs. We just put a L

> “GPT-3 is not necessarily well-calibrated in its predictions on novel inputs.” I don't thing even humans are well calibrated on novel inputs. We just put a LOT of effort via training into preventing inputs from being novel. Honest to God "novel inputs" break people too. Honestly I think the AGI problem will end with us admitting humans are a lot dumber than we like to believe they are more than it will require AGI getting smarter.