Am I missing something or would this be nearly impossible to track? You buy some bitcoin here and there and occasionally transfer some to a wallet from which y

Am I missing something or would this be nearly impossible to track? You buy some bitcoin here and there and occasionally transfer some to a wallet from which you pay for items. The price of bitcoin is rising and falling constantly, such that when you buy that cup of coffee, you could either be realizing a capital gain or loss of X amount. How on earth would you track this without losing your mind? And, how would the IRS enforce this?