And how do they do that (scan a web site directory tree) without following links? You are describing the same "following links" procedure I think. Maybe somebo

And how do they do that (scan a web site directory tree) without following links? You are describing the same "following links" procedure I think. Maybe somebody accessed those pages using the Google toolbar, in that case Google could have known about those pages even if they were not linked from any other pages elsewhere.