A software-only solution where private keys have the potential of leaving the device, be it an app or an USB-stick, is not really okay in this day and age. So

A software-only solution where private keys have the potential of leaving the device, be it an app or an USB-stick, is not really okay in this day and age. So a proper fallback wouldn't be an USB-stick with some certificates laying around, you can instead just build the ID functionality using SIM-card applets. The keys are securely stored in hardware and you can use them on both smart and feature phones.