3 ms·
because i may also come to the point where i am a direct competitor to google, but i will never get there because i can't scrap any site like they can. your ne
by profeta 10y ago
because i may also come to the point where i am a direct competitor to google, but i will never get there because i can't scrap any site like they can.
your next argument may very well be a very racist one with the very same excuse you used above.
- sjwright 10y agoAnd if you have some way to identify yourself as a potential competitor to google and not some jackass trying to scrape email addresses or spam comments forms, I'm all ears.
- greglindahl 10y agoA majority of the websites that blekko, a google competitor, contacted to ask for robots.txt access ignored us.
- sjwright 10y agoI agree, it's a difficult conundrum. It sucks.
- greglindahl 10y agoThere are worse barriers to entry for a search engine! DMCA take-downs... Right to be forgotten... click history...