I run my own mailserver and have done since ~2012. I don't have DKIM or DMARC set up but I am using postfix and not qmail. I'm sorry I can't help you with you

I run my own mailserver and have done since ~2012. I don't have DKIM or DMARC set up but I am using postfix and not qmail. I'm sorry I can't help you with your delivery problem - except that to say that if you didn't have eg rDNS set up to begin with gmail might have a negative cache of it. I haven't had problems with delivery of outgoing mail except to a couple of poorly administered exchange hosts run by recruitment agencies - which complain about my mail but confusingly do still deliver it. Two real problems I have faced...I once (embarrassingly) created a unix user with test/test credentials for messing about and forgot that my postfix setup at the time reused unix credentials (ssh was locked down to only allow specific user to log in). I sent a few million spams a hour for a couple of weeks, getting my host into all the DNS blocklists. This took some time to fix (you have to apply to have your host removed from the blacklist). While I'm sure sending all that spam was annoying to many other people it didn't actually affect delivery for my mail...so it seems other administrators aren't using DNS blocklists? Second, after a while I started to get a lot of spam. Maybe 10 per day. I tried various things to handle this, including setting up a proper bayesian spam filter (amavis-new) and using DNS blocklists myself. None of this worked for me. Greylisting however worked great. So my suggestions to you: use defence in depth for mail as well as ssh. That means, fail2ban, different creds for both, unusual ports, user whitelists, high patchlevels (auto-patch and restart is great for a personal mail server) maybe client side TLS certs...etc. If you're relying on a single layer of defence eventually you'll make a mistake with it and then you're in trouble. I guess that really applies to anything you're trying to secure.