When the other fills in the void of an awkward pause, it tends to be to try to further convince you to say yes, which can be really annoying. I usually just say

When the other fills in the void of an awkward pause, it tends to be to try to further convince you to say yes, which can be really annoying. I usually just say sorry, no by default, then I can change mind after if necessary.