Actually it does have something at stake: its singular goal of minimizing the loss function on its training data. AI is therefore designed to convince you it's

Actually it does have something at stake: its singular goal of minimizing the loss function on its training data. AI is therefore designed to convince you it's right, which is a different optimization than actually being right. For example, I code a lot with agentic code editors, you’ll quickly learn they love to modify broken tests to superficially pass, rather than fixing the underlying failure. All the folks on the Vibe Coding hype train don't have enough experience to spot this and therefore think the AI is a lot smarter than it actually is. This has scary implications if extrapolated out to extremes, because a sufficiently advanced AI would do exactly what you’re describing, manipulate power dynamics to get to a superficial outcome.