The effect might not even be physiological yet still be this prominent: having COVID-19 makes you doubly scared for your loved-ones, worry about your perception

The effect might not even be physiological yet still be this prominent: having COVID-19 makes you doubly scared for your loved-ones, worry about your perception in the society (was I not careful enough?), worry about lingering side-effects, worry about missing extended time from work, etc. While this depends on the individual, you can notice effects of increased stress in large numbers. Your suggestion of another control group with Covid-19-but-no-symptoms would help answer that question too, though ideally we'd want those who had Covid19 but didn't know it at all. But anyway, this paper is only establishing correlation: getting to causation in medicine is the harder task and takes years or decades and even then it's usually inconclusive!