There are in-kernel consumers of entropy (such as for aslr) that wouldn't show up as /dev/random opens. Reading urandom will deplete the available entropy, but

There are in-kernel consumers of entropy (such as for aslr) that wouldn't show up as /dev/random opens. Reading urandom will deplete the available entropy, but won't block - however, anything relying on actual entropy availability will then block until entropy is repopulated. So if entropy_avail is ever hitting 0, there's some vague plausibility that you could have some slowdowns. But honestly, it doesn't seem likely.