5 ms·

Google has multiple ways of indexing, and following links is only one of them. I suspect Google found these pages by scanning the directory structure of facebo

Google has multiple ways of indexing, and following links is only one of them. I suspect Google found these pages by scanning the directory structure of facebook, indexing all pages in these subdirectorys, and finding more subdirectories to drill in to.

eli 16y ago
I think (though I'm too lazy to confirm) that the Google Toolbar browser plugin also reports URLs back to Google
- sounddust 16y ago
  If Google is using their toolbar and/or Chrome to report private URLs back to Google for indexing, then that's a much more serious privacy violation than what's being reported here.
  - eli 16y ago
    A "private" URL should surely require a login or at least have a robots.txt, no?
    - masomenos 16y ago
      Plenty of sites use URLs like http://example.com/sd8n9g9g http://example.com/sd8n9g9g rather than require logins for private / semi-private material. What's wrong with that?
      - nl 16y ago
        It's a pretty bad practice. I guess for semi-private material it's okay, but otherwise that's liable to leak. The biggest hole in it is referrers leaking to other sites, but I can think of other methods (eg probing the CSS history exposure bug).
    - michaelfairley 16y ago
      Or a noindex <meta> tag
  - count 16y ago
    Which is why you aren't allowed to have Google Toolbar or Google Desktop installed on US Govt computers.
- nfriedly 16y ago
  The Google toolbar reports URLs back to Google if you have page rank or a few other features turned on. Their privacy policy says that they store the URL and provide you with whatever feature you requested, but it doesn't say weather they do anything further with the data. Relevant bits: URLs and embedded information Some of our services, including Google Toolbar and Google Web Accelerator, send the uniform resource locators (“URLs”) of web pages that you request to Google. When you use these services, Google will receive and store the URL sent by the web sites you visit, including any personal information inserted into those URLs by the web site operator. Some Google services (such as Google Toolbar) enable you to opt-in or opt-out of sending URLs to Google, while for others (such as Google Web Accelerator) the sending of URLs to Google is intrinsic to the service. When you sign up for any such service, you will be informed clearly that the service sends URLs to Google, and whether and how you can opt-in or opt-out. For example, when you submit information to a web page (such as a user login ID or registration information), the operator of that web site may “embed” that information – including personal information – into its URL (typically, after a question mark (“?”) in the URL). When the URL is transmitted to Google, our servers automatically store the URL, including any personal information that has been embedded after the question mark. Google does not exercise any control over these web sites or whether they embed personal information into URLs. http://www.google.com/privacy_faq.html#toc-terms-urls http://www.google.com/privacy_faq.html#toc-terms-urls Uses We process your requests in order to operate and improve the Google Toolbar and other Google services. For example, by knowing which web page you are viewing, the PageRank feature of Google Toolbar can show you Google's ranking of that web page. And the Sidewiki feature can tell you if others have written Sidewiki entries on a given page. Likewise, by processing the text on a web page, SpellCheck can offer spelling suggestions and AutoLink can provide useful links to information. http://www.google.com/support/toolbar/bin/answer.py?hl=en&answer=81841 http://www.google.com/support/toolbar/bin/answer.py?hl=en&#3...
loboman 16y ago
And how do they do that (scan a web site directory tree) without following links? You are describing the same "following links" procedure I think. Maybe somebody accessed those pages using the Google toolbar, in that case Google could have known about those pages even if they were not linked from any other pages elsewhere.

eli 16y ago

I think (though I'm too lazy to confirm) that the Google Toolbar browser plugin also reports URLs back to Google

sounddust 16y ago
If Google is using their toolbar and/or Chrome to report private URLs back to Google for indexing, then that's a much more serious privacy violation than what's being reported here.
- eli 16y ago
  A "private" URL should surely require a login or at least have a robots.txt, no?
  - masomenos 16y ago
    Plenty of sites use URLs like http://example.com/sd8n9g9g http://example.com/sd8n9g9g rather than require logins for private / semi-private material. What's wrong with that?
    - nl 16y ago
      It's a pretty bad practice. I guess for semi-private material it's okay, but otherwise that's liable to leak. The biggest hole in it is referrers leaking to other sites, but I can think of other methods (eg probing the CSS history exposure bug).
  - michaelfairley 16y ago
    Or a noindex <meta> tag
- count 16y ago
  Which is why you aren't allowed to have Google Toolbar or Google Desktop installed on US Govt computers.
nfriedly 16y ago
The Google toolbar reports URLs back to Google if you have page rank or a few other features turned on. Their privacy policy says that they store the URL and provide you with whatever feature you requested, but it doesn't say weather they do anything further with the data. Relevant bits: URLs and embedded information Some of our services, including Google Toolbar and Google Web Accelerator, send the uniform resource locators (“URLs”) of web pages that you request to Google. When you use these services, Google will receive and store the URL sent by the web sites you visit, including any personal information inserted into those URLs by the web site operator. Some Google services (such as Google Toolbar) enable you to opt-in or opt-out of sending URLs to Google, while for others (such as Google Web Accelerator) the sending of URLs to Google is intrinsic to the service. When you sign up for any such service, you will be informed clearly that the service sends URLs to Google, and whether and how you can opt-in or opt-out. For example, when you submit information to a web page (such as a user login ID or registration information), the operator of that web site may “embed” that information – including personal information – into its URL (typically, after a question mark (“?”) in the URL). When the URL is transmitted to Google, our servers automatically store the URL, including any personal information that has been embedded after the question mark. Google does not exercise any control over these web sites or whether they embed personal information into URLs. http://www.google.com/privacy_faq.html#toc-terms-urls http://www.google.com/privacy_faq.html#toc-terms-urls Uses We process your requests in order to operate and improve the Google Toolbar and other Google services. For example, by knowing which web page you are viewing, the PageRank feature of Google Toolbar can show you Google's ranking of that web page. And the Sidewiki feature can tell you if others have written Sidewiki entries on a given page. Likewise, by processing the text on a web page, SpellCheck can offer spelling suggestions and AutoLink can provide useful links to information. http://www.google.com/support/toolbar/bin/answer.py?hl=en&answer=81841 http://www.google.com/support/toolbar/bin/answer.py?hl=en&#3...

loboman 16y ago

And how do they do that (scan a web site directory tree) without following links? You are describing the same "following links" procedure I think. Maybe somebody accessed those pages using the Google toolbar, in that case Google could have known about those pages even if they were not linked from any other pages elsewhere.