"and if at all possible keep them off the public internet", this is the way. I would recommend going beyond a VPN to implement zero trust networking which does

"and if at all possible keep them off the public internet", this is the way. I would recommend going beyond a VPN to implement zero trust networking which does outbound-only connections so that its impossible to be subject to external network attacks. Tailscale does part of that, other exist, such as the open source project I work on - https://github.com/openziti https://github.com/openziti