4 ms·
I downloaded all my logs from a public-facing development server. Hitting anything returns "403 Forbidden" except for a few domains which should not be crawled
by ancarda 13y ago
I downloaded all my logs from a public-facing development server. Hitting anything returns "403 Forbidden" except for a few domains which should not be crawled (google does it anyway). Most traffic is usual (hit / and /robots.txt then leave). All I found (in 41 rotated logs) was:
access.log.12:66.249.73.132 - - [29/Oct/2013:01:26:15 +0100] "GET /?ac=2 HTTP/1.1" 403 135 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.12:66.249.73.132 - - [29/Oct/2013:03:24:12 +0100] "GET /?tag=lazy HTTP/1.1" 403 135 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.3:66.249.66.57 - - [06/Nov/2013:12:12:42 +0100] "GET /?ac=2&slt=8&slr=1&lpt=1 HTTP/1.1" 403 135 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.4:66.249.75.57 - - [05/Nov/2013:19:14:19 +0100] "GET /?cat=1 HTTP/1.1" 403 135 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.5:66.249.66.109 - - [04/Nov/2013:15:54:01 +0100] "GET /?ac=2&slt=8&slr=1&lpt=1 HTTP/1.1" 403 135 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.5:66.249.75.18 - - [05/Nov/2013:03:08:27 +0100] "GET /?ac=2&slt=8&slr=1&lpt=1 HTTP/1.1" 403 135 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
I have no idea what "?ac", "?cat" or "?tag" are suppose to do. Nothing on the server responds to GET params (I use url rewrites and POST only) so I don't think I ever made a link (not even accidentally).
I found nothing for "/profile" or "/news"
- parham 13y agoMy server is also a public-facing development server. I think they might be trying to find pages that aren't linked to directly in a hope to find more unlined pages... just a theory
- andrewcooke 13y agoi've found an example of a similar url - http://stackoverflow.com/questions/19674788/redirect-url-having-query-string-to-404-page-using-htaccess http://stackoverflow.com/questions/19674788/redirect-url-hav... - so i am wondering if you have an ip that previously belonged to someone using whatever software that is (some kind of index or pagination i assume) and which had links with a numeric ip address....
- parham 13y agoOh wow didn't think of that, that is the reason, the IP address I'm using is an Elastic IP from AWS that I'm reusing from an old project, that project had those exact same links.