The rate at which it was drifting with respect to the other stars might have been noticeable under long-term observations. 20 ly in 70k years means its doing a

The rate at which it was drifting with respect to the other stars might have been noticeable under long-term observations. 20 ly in 70k years means its doing about 85 km/s. At its point of closest approach, 0.6 ly, this translates to 1 radian per 2100 years, or 1 degree per 37 years. (The full moon is 1/2 a degree.) The Scholz star was too dim to see, but a brighter star following its trajectory would have noticeably moved in a constellation over the course of a human lifetime.