2 ms·
The first wave of the attack began dropping off letters. I wouldn't call it an attack, but it seems more like some sort of autocompletion checking given how si
by userbinator 8y ago
The first wave of the attack began dropping off letters.
I wouldn't call it an attack, but it seems more like some sort of autocompletion checking given how similar it is to the queries generated to Google if you use their search autocomplete feature --- one request for each character entered.
That said, Google's search quality has already taken a steep decline and I've been getting blocked very often for searching more obscure things now, so whatever "bot mitigations" they put in, I hope they don't make that even worse.
- wybiral 8y ago> I've been getting blocked very often for searching more obscure things Me too! It seems like nearly any time I go past the second page for results they think I'm a bot.
- _asummers 8y agoThe claim was that it started with the full URL and progressively removed them. If it were reversed I would buy the autocomplete bug. I fixed something like that a few weeks ago at work.
- mynameisvlad 8y agoIf Chrome address bar autocomplete generates queries (which I'm not sure if it does), then the following scenario could easily explain what's going on: - As a user, I type "foo.com" into my address bar. Having already gone to a specific page of that site before, autocomplete "helpfully" expands it to that page's URL "https://foo.com/this/is/a/page" https://foo.com/this/is/a/page" - It turns out I don't actually want that, I want the main page, so I start deleting characters until I'm left with "https://foo.com" https://foo.com" If Google took each of those deletions as a separate URL (either by bug or on purpose) and tried to verify said URL is valid, I could see a GoogleBot request being generated for each iteration between the initial URL and the full URL, starting with that initial one.
- estebank 8y agoGiven what I've read on the article, I'm fairly confident that what's happening here is a conjunction of Chrome autocomplete suggestions causing the crawl of every URL permutation and the website returning the same content to every permutation, instead of using 403 redirects or including a `rel=canonical tag`.