3 ms·
The money mentioned in that article is about being the default search engine, not being a functional one. Basically the question I'm asking is why DDG has to ou
by brundolf 6y ago
The money mentioned in that article is about being the default search engine, not being a functional one. Basically the question I'm asking is why DDG has to outsource to Bing.
> there are a lot of big sites that will forbid you from crawling their content, unless you are Google, or Bing
If you're talking about robots.txt, that's just a suggestion. It doesn't hold any actual preventative power.
- edoceo 6y agoIP blocking and rate limiting for clients outside the blessed range operated by G