After talking through this thread a bit, I realized that I'm pretty sure emauth.io already works the way you're describing. Having the link alone is not enough

After talking through this thread a bit, I realized that I'm pretty sure emauth.io already works the way you're describing. Having the link alone is not enough to login. Only the process (whether in browser, on your server, curl, or whatever) that made the original request will know that the email has been verified. The browser that opens the link simply gets a text response that says the email was verified (or failed if already expired). You can use that information however makes the most sense. For a secure application, it would be used by your server to validate a session.