This has been my experience here in Osaka. Even though there were dozens of other Japanese around, an old man walked up and asked me in Japanese if I knew where

This has been my experience here in Osaka. Even though there were dozens of other Japanese around, an old man walked up and asked me in Japanese if I knew where the closest ATM for his particular bank was. I'm not entirely sure what it is, but I think you eventually hit a threshold where you are somehow signaling via body language or manner that, as you say, you don't need to be handled like an outsider. Edit: I just noticed I have replied to you twice. We seem to be on the same page :)