The census has an official date to avoid issues like this. Since they lived at the house across the street from you on the official census date of April 1st 202

The census has an official date to avoid issues like this. Since they lived at the house across the street from you on the official census date of April 1st 2020, the census worker was correct to record them at that address. They would only have been double counted if they incorrectly reported living at their new address as of April 1st.