5 ms·

The way I take that to mean is: if you're using a Googlebot UA, and your request doesn't come from a Google-owned IP range, you're very unlikely to be a legit c

The way I take that to mean is: if you're using a Googlebot UA, and your request doesn't come from a Google-owned IP range, you're very unlikely to be a legit crawl from Google, and you'll be detected and dealt with.

Crosseye_Jack 7y ago
Which would be fine imo. But as the person I was replying to says you can already detect a bad bot pretending to be google and deal with them how you wish (my fav is just feeding them random values). But what are CF (and others like akamai) using to determine you are a bad bots? My objection is more the threat they will use their position to kick bad bots offline. Now don't get me wrong, I had bad bots as much as the next person. But what about misfires? They will happen. I hope CF have a plan in place to deal with such cases. Personally, due to family issues I've been working remote-remote (I'm a remote worker anyways, but I've been working remotely away from my normal office). As such I've been using a 4G Mobile broadband connection and the amount of CloudFlare "We are just checking you are not a bot" hoops I've had to jump though recently has opened my eyes on how much of the net is actually behind CF's network because CF still associate "IP as Identity" and my 4G connection is CG-NAT'ed. I just fear at times we forget that there is the rest of the world out there, not just our little bubble and the choices we make have a huge impact on the usability of the net for others. EDIT: LOL.. Just checked your profile. The Reg is one of the sites that doesn't like my 4G connection and as such I visit less. What are the chances. (Though you only make me sit there and wait to access the content....)
- mfontani 7y ago
  That's odd, TBQH. Contact webmaster@ with some details about the connection (your IP at the time you're seeing the problem, as well as your UA - all stuff you can get by visiting /cdn-cgi/trace) and I'll see if there's anything we can tweak on our end. No promises tho!
  - Crosseye_Jack 7y ago
    I'm CG-NAT'ed... There is no guarantee from my ISP that 2 requests to the same site in a short period of time will actually come from the same public IP address. But I might next time I have the urge to visit the reg, because honestly waiting for CF's permission to visit your site has reduced my patience to access the content so I go elsewhere and don't even bother clicking on links to the reg any more, So I honestly don't know if I will be arsed to jump though more hoops just to be told "Your CG-NAT'ed... What else do you want than allowing your whole ISP free rein to the site without having to spin CPU cycles just in-case you are a bot." (because what else can you say?) Sorry if that's not the answer you wanted, but that's how I feel after seeing how the other side of the net is treated. IF the urge takes me, I'll help, but I dunno when that urge will take me though. <EDIT>Fun Fact: My public IP address appears to have changed in the time it took me to write this comment. Atleast to the reg, How my ISP handles routing is another black box upon itself.</EDIT>

Crosseye_Jack 7y ago

Which would be fine imo. But as the person I was replying to says you can already detect a bad bot pretending to be google and deal with them how you wish (my fav is just feeding them random values). But what are CF (and others like akamai) using to determine you are a bad bots? My objection is more the threat they will use their position to kick bad bots offline. Now don't get me wrong, I had bad bots as much as the next person. But what about misfires? They will happen. I hope CF have a plan in place to deal with such cases. Personally, due to family issues I've been working remote-remote (I'm a remote worker anyways, but I've been working remotely away from my normal office). As such I've been using a 4G Mobile broadband connection and the amount of CloudFlare "We are just checking you are not a bot" hoops I've had to jump though recently has opened my eyes on how much of the net is actually behind CF's network because CF still associate "IP as Identity" and my 4G connection is CG-NAT'ed. I just fear at times we forget that there is the rest of the world out there, not just our little bubble and the choices we make have a huge impact on the usability of the net for others. EDIT: LOL.. Just checked your profile. The Reg is one of the sites that doesn't like my 4G connection and as such I visit less. What are the chances. (Though you only make me sit there and wait to access the content....)

mfontani 7y ago
That's odd, TBQH. Contact webmaster@ with some details about the connection (your IP at the time you're seeing the problem, as well as your UA - all stuff you can get by visiting /cdn-cgi/trace) and I'll see if there's anything we can tweak on our end. No promises tho!
- Crosseye_Jack 7y ago
  I'm CG-NAT'ed... There is no guarantee from my ISP that 2 requests to the same site in a short period of time will actually come from the same public IP address. But I might next time I have the urge to visit the reg, because honestly waiting for CF's permission to visit your site has reduced my patience to access the content so I go elsewhere and don't even bother clicking on links to the reg any more, So I honestly don't know if I will be arsed to jump though more hoops just to be told "Your CG-NAT'ed... What else do you want than allowing your whole ISP free rein to the site without having to spin CPU cycles just in-case you are a bot." (because what else can you say?) Sorry if that's not the answer you wanted, but that's how I feel after seeing how the other side of the net is treated. IF the urge takes me, I'll help, but I dunno when that urge will take me though. <EDIT>Fun Fact: My public IP address appears to have changed in the time it took me to write this comment. Atleast to the reg, How my ISP handles routing is another black box upon itself.</EDIT>