3 ms·

Google is crawling all over the web but cannot detect these as fraudulent requests? That seems like an easy check. Do we have an index of this domain? Was this

by ohashi 8y ago

Google is crawling all over the web but cannot detect these as fraudulent requests? That seems like an easy check. Do we have an index of this domain? Was this backdated? But they could be blocking it with robots.txt or some other excuse. It seems like an incredibly rare exception, what news site would block being crawled and indexed? They also are archiving domain info I'm sure. If they wanted to stop this, they have the data.

jobigoud 8y ago
> That seems like an easy check. Do we have an index of this domain? Was this backdated? How do you know it's backdated if by the time you crawl the articles have appeared on both sides. How do you know which was published first? This attack can be performed faster than the re-crawling frequency.
- ohashi 8y ago
  It could be, but from the article that came out, it seems like it's a lot slower. This example was over a year later.
iliketosleep 8y ago
They surely have both the capability and data to thwart this type of fraud. There are only two possibilities that I can think of. One is that it's a business decision, where they know these cases may exist but they took the decision that it's not worth the extra overhead. Second possibility is that it's an oversight, essentially a "bug" in the system. Either way, it appears that Google had not yet identified this as a serious problem. I give credit to them for having a Transparency Report so that others can pick up fraud and abuse which they themselves may not have already found to have been a problem; an essential safeguard. I also view projects such as Lumen as being of increasing importance.
- amygdyl 8y ago
  if I suspected my services were being manipulated by fraudulent activities, for potential political influence, I might just pay out enough rope for fraudsters to hang themselves by. if I really hated holding the poison chalice to be the arbiter of so much public credulity, I think, possibly, I might not react to this kind of publicity about my apparent oversight, to see who I could catch..
Ajedi32 8y ago
I don't know, is "we've never seen this page on your site before" a valid defense against a DMCA claim? If I understand DMCA correctly, ignoring the claim means risking a lawsuit if you're wrong about the content not being owned by the claimant.
- amygdyl 8y ago
  I would ask for a documentary chain of evidence in support of the claimed authorship.
- markdown 8y ago
  I lost a domain a while ago. It was quickly snapped up by a squatter, who promptly repopulated it with the exact content it previously contained (harvested from the waybackmachine). They made a single change to the content, adding a link on the home page to some supplement blog. So it's my content, on a domain I no longer own or control. How could I prove to Google that the content is mine?
  - slededit 8y ago
    DMCA doesn't require proof. The fact you are attesting under penalty of perjury is enough. If they contest it then your only remaining remedy is a court order and that would require proof. The wayback machine may be sufficient.
  - kilovoltaire 8y ago
    I recently learned that this is due to a somewhat shady SEO trick referred to as Private Blog Networks. You take over expiring legitimate domains, and ideally immediately (before Google notices) put up almost the same content, but with a link to whatever site you're actually interested in giving better search ranking to. Repeat this 1000x and you can apparently make enough headway that people put time and money into the approach.
reaperducer 8y ago
Google is crawling all over the web but cannot detect these as fraudulent requests? Google crawls surprisingly little of the web, even on web sites it knows about. That’s why web developers have to use site indices and Google Webmaster Tools to tell G that stuff exists on their sites, hoping that it will be discovered and indexed.

by ohashi 8y ago

jobigoud 8y ago

> That seems like an easy check. Do we have an index of this domain? Was this backdated? How do you know it's backdated if by the time you crawl the articles have appeared on both sides. How do you know which was published first? This attack can be performed faster than the re-crawling frequency.

ohashi 8y ago
It could be, but from the article that came out, it seems like it's a lot slower. This example was over a year later.

iliketosleep 8y ago

They surely have both the capability and data to thwart this type of fraud. There are only two possibilities that I can think of. One is that it's a business decision, where they know these cases may exist but they took the decision that it's not worth the extra overhead. Second possibility is that it's an oversight, essentially a "bug" in the system. Either way, it appears that Google had not yet identified this as a serious problem. I give credit to them for having a Transparency Report so that others can pick up fraud and abuse which they themselves may not have already found to have been a problem; an essential safeguard. I also view projects such as Lumen as being of increasing importance.

amygdyl 8y ago
if I suspected my services were being manipulated by fraudulent activities, for potential political influence, I might just pay out enough rope for fraudsters to hang themselves by. if I really hated holding the poison chalice to be the arbiter of so much public credulity, I think, possibly, I might not react to this kind of publicity about my apparent oversight, to see who I could catch..

Ajedi32 8y ago

I don't know, is "we've never seen this page on your site before" a valid defense against a DMCA claim? If I understand DMCA correctly, ignoring the claim means risking a lawsuit if you're wrong about the content not being owned by the claimant.

amygdyl 8y ago
I would ask for a documentary chain of evidence in support of the claimed authorship.
markdown 8y ago
I lost a domain a while ago. It was quickly snapped up by a squatter, who promptly repopulated it with the exact content it previously contained (harvested from the waybackmachine). They made a single change to the content, adding a link on the home page to some supplement blog. So it's my content, on a domain I no longer own or control. How could I prove to Google that the content is mine?
- slededit 8y ago
  DMCA doesn't require proof. The fact you are attesting under penalty of perjury is enough. If they contest it then your only remaining remedy is a court order and that would require proof. The wayback machine may be sufficient.
- kilovoltaire 8y ago
  I recently learned that this is due to a somewhat shady SEO trick referred to as Private Blog Networks. You take over expiring legitimate domains, and ideally immediately (before Google notices) put up almost the same content, but with a link to whatever site you're actually interested in giving better search ranking to. Repeat this 1000x and you can apparently make enough headway that people put time and money into the approach.

reaperducer 8y ago

Google is crawling all over the web but cannot detect these as fraudulent requests? Google crawls surprisingly little of the web, even on web sites it knows about. That’s why web developers have to use site indices and Google Webmaster Tools to tell G that stuff exists on their sites, hoping that it will be discovered and indexed.