See my comment here: https://news.ycombinator.com/item?id=38336374 https://news.ycombinator.com/item?id=38336374 The reason AI alignment is challenging is beca

See my comment here: https://news.ycombinator.com/item?id=38336374 https://news.ycombinator.com/item?id=38336374 The reason AI alignment is challenging is because we're trying to make accurate predictions about unusual scenarios that we have essentially zero data about. No one can credibly claim expertise on what would constitute evidence of a worrisome anomaly. Jeremy Howard can't credibly say that a sudden drop in the loss function is certainly nothing to worry about, because the entire idea is to think about exotic situations that don't arise in the course of ordinary machine learning work. And the "loss" vs "loss function" thing is just silly gatekeeping, I worked in ML for years -- serious people generally don't care about minor terminology stuff like that.