This is an important area for AI safety research; see the ELK paper for example. https://www.alignmentforum.org/posts/qHCDysDnvhteW7kRd/arc-s-first-technical-r

This is an important area for AI safety research; see the ELK paper for example. https://www.alignmentforum.org/posts/qHCDysDnvhteW7kRd/arc-s-first-technical-report-eliciting-latent-knowledge https://www.alignmentforum.org/posts/qHCDysDnvhteW7kRd/arc-s... That paper is a bit dense, but considers the ways that a powerful AI model could be intractable/deceptive to discovering its latent knowledge. If we can confidently understand an AI’s internal knowledge/intention states, then alignment is probably tractable.