Out of curiosity, what would be a motivation for not including a non-fee-paying transaction in the block that you are trying to generate? I mean, theoretically

Out of curiosity, what would be a motivation for not including a non-fee-paying transaction in the block that you are trying to generate? I mean, theoretically, for each transaction that you are including in the block, the longer it would take to hash the block with the appropriate nonce (I believe that's what the 'extra random number' is called?), but is that the only reason for not including them? ie. the less transactions, the less time it takes to test each nonce? I seem to understand the basics of BTC... it's the deeper, sometimes less technical, concepts that trip me up.