I see common milkweed and swamp milkweed all over the place around here (upstate NY). I have swamp milkweed in my garden (deliberately planted, it's attractive

I see common milkweed and swamp milkweed all over the place around here (upstate NY). I have swamp milkweed in my garden (deliberately planted, it's attractive; I prefer native plants.) I have never seen a Monarch caterpillar around here in the five years I've lived here. I don't think host plant availability is the problem here. In an opposite situation: when I planted Pearly Everlasting the thing was eaten the ground by (non-Monarch) caterpillars. A bit too much of a good thing! It survived, thankfully.