4 ms·
I noticed this as well a couple of weeks ago. They're still actively visiting every page on the websites that are associated with our iOS apps. Today alone (s
by christop 12y ago
I noticed this as well a couple of weeks ago.
They're still actively visiting every page on the websites that are associated with our iOS apps.
Today alone (starting shortly before 8am CET) they've crawled over 8000 pages on https://trails.io https://trails.io and http://offmaps.com http://offmaps.com — without sending conditional caching headers (our pages don't change too often).
$ grep ^17\\. /var/log/nginx/*.log | grep 'Go|Fetcher' | wc -l
8254
Note that they don't appear to be scraping any URLs available from within the apps themselves, but rather the the company/support websites linked in our App Store listings.
I guess they're automatically scanning for objectionable content, since these websites are linked from the App Store and the iTunes website?
- 0x0 12y agoOoh, that's interesting. They already increased the requirements for online privacy policies etc for ios8 apps, so perhaps we'll see some crackdown on appstore apps who are missing those.