OP's comparison wasn't a homemade filter versus no filter, it was versus commercial options. I've done box fan filters before, and they certainly help if they'

OP's comparison wasn't a homemade filter versus no filter, it was versus commercial options. I've done box fan filters before, and they certainly help if they're what you can afford or can find in stores, but after enough years of dealing with wildfire smoke this year I invested in commercial units and I am significantly happier. After doing a fair amount of research, I ended up with a series of Coway units. They're more expensive than box fan filters, but they're smaller and significantly quieter, especially with a sensor-driven auto mode that adjusts fan speed based on detected pollution levels. This means I'm far more willing to leave them on all the time and get the benefits 24/7. The fact that they're multi-stage is a big plus too, particularly with cat hair around. It means that I have to clean the outer charcoal filters more often but the inner HEPA filters only 1-2 times a year. As an added bonus, power consumption is significantly lower than my box fans.