I've seen PAM modules for otp/totp, but that really only adds security if authenticating against a remote machine, if you validate the second factor locally doe

I've seen PAM modules for otp/totp, but that really only adds security if authenticating against a remote machine, if you validate the second factor locally doesn't that still open you to mitm or other exploits? Can you trust your own environment?