4 ms·
> Google use your personal data to put you in a bubble of your own interests. Creating illusion of objectivity, it narrows our perspective down and slows down t
by kazinator 7y ago
> Google use your personal data to put you in a bubble of your own interests. Creating illusion of objectivity, it narrows our perspective down and slows down the process of personal development.
I don't agree. People want this. Long before computers, let alone Google, people had no problem in absorbing themselves in themselves and their interests.
But, wait, really? Google uses my personal data to put me in a bubble of my own interests? A large chunk of the results for anything I search for are SEO-d garbage and many are irrelevant in other ways. That's supposed to be a bubble consisting of my interests?
> Google ignores websites privacy. Even if the website owner doesn't want to expose any content for search engines, Google will come, get it, and show it on its search results and will make money off it.
Absolute, unadulterated nonsense. The Google spider identifies itself via HTTP headers. It's trivial to refuse service to specific clients.
Example log:
66.249.79.219 - - [23/Jun/2019:15:36:58 -0700] "GET /robots.txt HTTP/1.1" 200 66 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
I could easily set up a filter to turn that into a 404 based on the request coming from Googlebot. Also note that it's accessing robots.txt. I don't have a link to that anywhere; it is spontaneously doing so; why would they if they had no intention of honoring the content.
I sometimes put private content on a URL to make it easily from somewhere. No links to it exist; you have to know the URL. I've never had such a thing accidentally indexed.
If people leave documents in searchable directories, don't use robots.txt and don't filter out the Google indexer, they shouldn't be surprised if those documents end up indexed.
- behindai 7y ago> Google ignores websites privacy. Even if the website owner doesn't want to expose any content for search engines, Google will come, get it, and show it on its search results and will make money off it. Absolute, unadulterated nonsense. The Google spider identifies itself via HTTP headers. It's trivial to refuse service to specific clients. Google spider ignores rules in robots.txt and show results from closed directories. I could write a distinct article about how we fight against it.
- kazinator 7y agoIf that is so why does the indexer bother fetching these files at all? It's a waste of bandwidth and cycles to be doing this extra fetch of robots.txt for countless sites.
- behindai 7y agoKidding me? The average size of robots.txt is 500 bytes for one web site! Average web page size is 2Mb. So if a crawler respectfully downloads a robots.txt it could significantly save more bandwidth not downloading protected pages.