Well with ssh usually you need to configure which keys are allowed per user so maybe `~/.tailpam_authorized` with either user, group, or ip (similar to tailscal

Well with ssh usually you need to configure which keys are allowed per user so maybe `~/.tailpam_authorized` with either user, group, or ip (similar to tailscale config) could control access per user. Alternatively, now that it's authenticated securely enabling password for authz could work maybe?