3 ms·

I don't believe this is a problem a crawler should be solving. The site left open SQL injection vulnerabilities, and they're complaining that someone is attacki

I don't believe this is a problem a crawler should be solving. The site left open SQL injection vulnerabilities, and they're complaining that someone is attacking them. Whether the attack originates from a crawler or not is beside the point.

ChuckMcM 13y ago
I agree that crawlers can't solve the problem, the trick is trying to avoid contributing to the problem. Take another well known example, DNS reflection attacks. Clearly DNS can't solve the DDOS "problem" but it can avoid helping out by not recursively resolving queries for untrusted clients. So any time the crawler can be a bit more aware about what it is heading into, the saves it time not looking at pages which we know apriori we would not be indexing anyway. And that (which pages should be indexed) can only be answered from the set of pages that are crawled.
garethadams 13y ago
The article is about an attacker using Google's behaviour as a form of indirection. The attacker can't be traced directly by the vulnerable site if he carried out the attack simply by triggering Google's crawler.
- jsmeaton 13y ago
  I agree that it mentions indirection, but I don't agree that it is the purpose of the article at all. The article finishes with this line: > We are contacting Google about it, but it is always something important to keep in the back of your mind. You can’t just whitelist their IP, and allow through without any type of inspection. I wouldn't be surprised if the authors knew of the sqli vector, and whitelists appropriate clients. The problem is the sqli, not WHO can hit it (or who was ultimately responsible). Edit: I just went and looked at the company behind this article. I was too quick to judge apparently. Their business is protecting developers from stupid mistakes like sqli at the firewall. Therefore, the article really IS about indirection. Sorry.

ChuckMcM 13y ago

I agree that crawlers can't solve the problem, the trick is trying to avoid contributing to the problem. Take another well known example, DNS reflection attacks. Clearly DNS can't solve the DDOS "problem" but it can avoid helping out by not recursively resolving queries for untrusted clients. So any time the crawler can be a bit more aware about what it is heading into, the saves it time not looking at pages which we know apriori we would not be indexing anyway. And that (which pages should be indexed) can only be answered from the set of pages that are crawled.

garethadams 13y ago

The article is about an attacker using Google's behaviour as a form of indirection. The attacker can't be traced directly by the vulnerable site if he carried out the attack simply by triggering Google's crawler.

jsmeaton 13y ago
I agree that it mentions indirection, but I don't agree that it is the purpose of the article at all. The article finishes with this line: > We are contacting Google about it, but it is always something important to keep in the back of your mind. You can’t just whitelist their IP, and allow through without any type of inspection. I wouldn't be surprised if the authors knew of the sqli vector, and whitelists appropriate clients. The problem is the sqli, not WHO can hit it (or who was ultimately responsible). Edit: I just went and looked at the company behind this article. I was too quick to judge apparently. Their business is protecting developers from stupid mistakes like sqli at the firewall. Therefore, the article really IS about indirection. Sorry.