I actually offered to refund most of the payment (we're still charged transaction fees for past payments), but before he answered me he disputed the transaction

I actually offered to refund most of the payment (we're still charged transaction fees for past payments), but before he answered me he disputed the transaction. At that point, it's too late and we aren't able to refund the transaction because it's been disputed.