You actually do both. Each measurement induces a probability distribution over the space upon which you bruteforce. Also not that timing attacks are quadratic

You actually do both. Each measurement induces a probability distribution over the space upon which you bruteforce. Also not that timing attacks are quadratic in the noise to signal ratio and linear in key length which bruteforce is exponential in keylengths. So that threshold is likely many orders of magnitude smaller than you imagine.