4 ms·
Is the article really talking about crawling? Because in one of their screenshots where they ask information about the "honeypot" website you can see that the m
by st3fan 1y ago
Is the article really talking about crawling? Because in one of their screenshots where they ask information about the "honeypot" website you can see that the model requested pages from the website. But that is most definitely "fetching by proxy because I asked a question about the website" and not random crawling.
It is confusing.
- johnfn 1y agoYea, now I feel like my comment might be misleading. The title mentions crawling; the article itself is talking about something else.