I'm not sure if it would have small to midsize towns like you are looking for, but Nomadlist might be a good place to start as it is a curation of popular place

I'm not sure if it would have small to midsize towns like you are looking for, but Nomadlist might be a good place to start as it is a curation of popular places for nomads (and no by extension remote workers) to relocate to. It uses a LOT of filters to weed out places you don't want and find the places that hit your "must haves" so it might be worth a quick look.