3 ms·

No, Google doesn't use deceptive means. They identify their crawler as GoogleBot, and obey robots.txt.

by jefftk 10mo ago

Nextgrid 10mo ago
Google doesn't have to do that now after already having established its own monopoly... just like SerpApi wouldn't have to act deceptively if they had a monopoly on search.
jppope 10mo ago
What about for their LLM products? We know that OpenAi does not respect the robots.txt file
- xnx 10mo ago
  Google uses the same crawler and robots.txt file for training data.
  - inkysigma 10mo ago
    It's actually a different crawler for training data: Googlebot-extended so you can exclude yourself from the training data though not the search summaries.
deleted 10mo ago
[deleted]
AstroBen 10mo ago
Because they've forced everyone to allow them. They're the internet traffic mafia. Block them and you disappear from the internet They abuse this power to scrape your work, summarize it and cut you out as much as possible. Pure value extraction of others' work without equal return. Now intensified with AI But yeah, you're right. They're not deceptive
- bitpush 10mo ago
  > Because they've forced everyone to allow them. nobody is forcing anyone. This is the same argument that people said about google search. Nobody is forcing anyone to use google search, google chrome, or even allow googlebot for scraping. Thousands of poeple have switched over to chatgpt, brave/firefox .. Your argument sounds like "I dont like Apple's practices, and I'm forced to buy iPhones. No buddy, if you dont like Apple, dont buy their products"
  - AstroBen 10mo ago
    > Your argument sounds like "I dont like Apple's practices, and I'm forced to buy iPhones. No buddy, if you dont like Apple, dont buy their products" No, not really. There are alternatives to Apple. Whereas here Google controls the gate to the majority of internet traffic For many it's "block Google and your business dies"
  - deleted 10mo ago
    [deleted]
  - thayne 10mo ago
    > Thousands of poeple have switched over to chatgpt, brave/firefox .. If you want people to visit your website, limiting yourself to the "thousands" of people who don't use google isn't really an option. > Your argument sounds like "I dont like Apple's practices, and I'm forced to buy iPhones. No buddy, if you dont like Apple, dont buy their products" Well, I don't like Apple's or Google's practices, but I basically [1] have to use either iOS or Android. [1]: yes there are things like GrapheneOS and librem, but those aren't really practical for most people.
deleted 10mo ago
[deleted]

No, Google doesn't use deceptive means. They identify their crawler as GoogleBot, and obey robots.txt.

by jefftk 10mo ago

No, Google doesn't use deceptive means. They identify their crawler as GoogleBot, and obey robots.txt.

Google doesn't have to do that now after already having established its own monopoly... just like SerpApi wouldn't have to act deceptively if they had a monopoly on search.

jppope 10mo ago

What about for their LLM products? We know that OpenAi does not respect the robots.txt file

xnx 10mo ago
Google uses the same crawler and robots.txt file for training data.
- inkysigma 10mo ago
  It's actually a different crawler for training data: Googlebot-extended so you can exclude yourself from the training data though not the search summaries.

deleted 10mo ago

[deleted]

AstroBen 10mo ago

Because they've forced everyone to allow them. They're the internet traffic mafia. Block them and you disappear from the internet They abuse this power to scrape your work, summarize it and cut you out as much as possible. Pure value extraction of others' work without equal return. Now intensified with AI But yeah, you're right. They're not deceptive

bitpush 10mo ago
> Because they've forced everyone to allow them. nobody is forcing anyone. This is the same argument that people said about google search. Nobody is forcing anyone to use google search, google chrome, or even allow googlebot for scraping. Thousands of poeple have switched over to chatgpt, brave/firefox .. Your argument sounds like "I dont like Apple's practices, and I'm forced to buy iPhones. No buddy, if you dont like Apple, dont buy their products"
- AstroBen 10mo ago
  > Your argument sounds like "I dont like Apple's practices, and I'm forced to buy iPhones. No buddy, if you dont like Apple, dont buy their products" No, not really. There are alternatives to Apple. Whereas here Google controls the gate to the majority of internet traffic For many it's "block Google and your business dies"
- deleted 10mo ago
  [deleted]
- thayne 10mo ago
  > Thousands of poeple have switched over to chatgpt, brave/firefox .. If you want people to visit your website, limiting yourself to the "thousands" of people who don't use google isn't really an option. > Your argument sounds like "I dont like Apple's practices, and I'm forced to buy iPhones. No buddy, if you dont like Apple, dont buy their products" Well, I don't like Apple's or Google's practices, but I basically [1] have to use either iOS or Android. [1]: yes there are things like GrapheneOS and librem, but those aren't really practical for most people.

deleted 10mo ago

[deleted]