It might not be obvious but I found that even though my profile is not "public", visitors who stumbled upon my Google Profile page could see who I was following

It might not be obvious but I found that even though my profile is not "public", visitors who stumbled upon my Google Profile page could see who I was following / who was following me. In the Edit Profile section, looks like "Display the list of people I'm following and people following me" is enabled by default.