We may notice some small things, but that doesn't mean that we've found all large things. According to https://www.nasa.gov/planetarydefense/faq https://www.na

We may notice some small things, but that doesn't mean that we've found all large things. According to https://www.nasa.gov/planetarydefense/faq https://www.nasa.gov/planetarydefense/faq we only know about 40% of the rocks sized 140 meters+ that cross Earth's orbit. If one of the 60% we don't know about came at us from the direction of the Sun, our odds of seeing it before impact are fairly small.