Well if you want to look at specific attack vectors this one may have remote access issues based on what and where you store the source code. I'd guess OP knows

Well if you want to look at specific attack vectors this one may have remote access issues based on what and where you store the source code. I'd guess OP knows what he's doing, but someone trying this with an accidentally open repo or comprised machine are possibly bigger risks here then with most other TOTP solutions. Equally it has the hardware key flaw of being able to be physically stolen, but with no option of an additional lock, and more likely then most systems that you might leave the totp running so a camera exploit is a little easier then with an app maybe. Not to say I think any of this is likely, and with the exception of a public repo mistake, it's probably a lot harder then an SMS exploit.