Debouncing only has to carry a latency penalty for certain kinds of responses. A good UI should be able to start processing an action as soon as the button goes

Debouncing only has to carry a latency penalty for certain kinds of responses. A good UI should be able to start processing an action as soon as the button goes down, and add/amend the actions later once it's clear whether it was a press and hold, press and release, or double-click. If the appropriate responses to those three options are mutually exclusive to the point that the UI can't respond at all until the user input is fully disambiguated, then the keybindings are probably ill-considered and those unrelated or conflicting functions should not be sharing a button.