5 ms·

I've had something like this happen to a customer of mine. Some of his content posts were being copied by random sketchy sites, then backdated by a couple hours

I've had something like this happen to a customer of mine. Some of his content posts were being copied by random sketchy sites, then backdated by a couple hours. Google would show the copied content as the main link in Google news, and his actual article would be buried somewhere under "more like this" link (or whatever it was called in news).

timClicks 8y ago
Really interesting, this behavior implies that Google hasn't figured out a way to verify what a page states as its creation date.
- askmike 8y ago
  Google crawls the open web non stop. They definitely have this data. It's just not exposed to the (legal) team dealing with DMCA takedown requests.. EDIT: Or as to the example above: whatever application is responsible for picking the main article to show somewhere.
  - codetrotter 8y ago
    They crawl the web non-stop yes, but how often an individual site is crawled varies. > Our crawl process is algorithmic: computer programs determine how often we should crawl each site. If you post new articles on your site throughout the day, you should be able to see them appear on Google News fairly quickly. > [...] > From the moment we discover a new article, we'll keep re-crawling it looking for changes. Since we noticed that most changes to articles occur just after they're published, we revisit articles most frequently in the first day after we've found them. After that, we visit them less often. https://support.google.com/news/publisher-center/answer/40392?hl=en https://support.google.com/news/publisher-center/answer/4039...
    - askmike 8y ago
      Fair point, if your content gets copied to unknown website X which Google hasn't crawled for days they might not be able to tell where the article appeared first.
- IncRnd 8y ago
  Have you figured out a way to verify what a page states as its creation date?
  - PeterisP 8y ago
    Ignore the claimed creation date, and use the earliest date that you know for certain, i.e. the time you crawled it.
    - jacquesm 8y ago
      That only works if you happen to crawl the original first. Given that copies could be plentiful that's a losing game. Content creators signing their content with a private key would be a better idea.
      - icebraining 8y ago
        Content creators signing their content with a private key would be a better idea. How would that help? Wouldn't the copier simply sign with their own key? Maybe Google should run a timestamping service. It could be stateless: send an hash of the content, get it back timestamped and signed, then include it in the page before publishing it.
        jacquesm 8y ago
        They could, but it would not be registered before the other one would be. - create signature - post signature, possibly publicly, (wow, maybe an actual use for a blockchain) - post content Anybody copying it could be proven to be a copyist.
        icebraining 8y ago
        Right, you'd need something like the blockchain - ie, a timestamping server - to validate the posting date. But then you don't need a signature, just a hash. The advantage of using Google's timestamping service is that it wouldn't need to store the hash/signature, unlike the blockchain.
- dexen 8y ago
  This could be solved with cryptographic timestamps [1]. The current generation of tooling is a bit raw - cli utility and Python library - but fully usable on the publisher's end. If the practice picked up any traction, surely more front-end tooling would be created. Example workflow: - write a blogpost - build a canonical representation based on a simple HTML or XML template; including canonical URL, title, content, authorship (emails, social media URLs, etc.), optionally related multimedia in a Zip archive, perhaps inlined via data URL - alternatively, re-purpose a common Web scrapping format, like the one used by Firefox, for canonical representation - submit the canonical representation for timestamping [2] - provide a <meta name="publication-timestamp" value="canonical representation url"> in the document as published, to facilitate automatic and manual verification - upon dispute, provide the canonical representation and verify in the relevant timestamping services [1] for example http://opentimestamps.org/ http://opentimestamps.org/ [2] from experience, the processing takes under 30 minutes with OpenTimeStamps
  - gambler 8y ago
    I agree that we need cryptographic timestamps. Seems like this would be a good use for blockchain, but any implementation would be better than nothing, as long as it's not fully centralized. The proposed workflow seems overly complex, though. For authorship verification all you need is ability to submit a block of text and receive a signed timestamp. Input: text Output: timestamp + signature Verification input: text + timestamp + signature. The simpler it is, the more universally it will be used. It should not 20 minutes either. The crypto involved should be pretty trivial. Any metadata could be built into the text itself as comments or hidden fields. However, this will not verify the URL the data is coming from, which might be desirable. The second kind of service could be verifiable archival service. Input: URL. Output: page content after rendering + timestamp + signature. Verification input: text + timestamp + signature. The service stores: timestamp + signature. This could be used to verify that at time X URL Y contained text Z. It doesn't matter whether the text is dynamic and go change later, since you could keep original page content.
    - dexen 8y ago
      > It should not [take?] 20 minutes either. Fair enough; the tech I know and mentioned (OpenTimestamps) is tied indirectly to Bitcoin's blockchain confirmation speed, thus both slow-ish, and also probabilistic rather than guaranteed. On the other hand, it gives good assurance that the tool will be around for a long time, and able to survive any single service provider's failure. > The crypto involved should be pretty trivial. Depends on the requirements. The tech I know and use (OTS) is privacy-conscious and handles timestamping of documents without revealing their actual content. Instead you submit hash(hash(document) + random, private IV) [1], which provides for both anonymous use, and ability to keep the signed content private, as the signed hash cannot be directly tied to any plaintext. This isn't strictly necessary for publicly-posted web pages, but certainly is useful for private documents, security-sensitive documents, etc. All in all, it's a starting point. Another possible approach would be to standardize submission to, and verification through a service like https://archive.is/ https://archive.is/, but that puts all trust into one 3rd party. > Input: [plain]text Directly timestamping the plaintext fails badly if the timestamping service provider is dishonest and creates an earlier timestamp for the content in their name, only to turn around and accuse the author of stealing the content. Or - a slightly paranoid scenario, but probably warranted in the post-Snowden era - if a whistleblower tries to timestamp some materials before distribution, but the timestamping service provider's filters catch the document and instead alert the government. Ergo, I'd prefer a blind timestamp, with the ability to reveal at the discretion of the author or whistleblower, even at the cost of more complex workflow. [1] sadly this means the author needs to keep a file containing the IV and a bit of other metadata.
johnchristopher 8y ago
What did you and the client do then ?

timClicks 8y ago

Really interesting, this behavior implies that Google hasn't figured out a way to verify what a page states as its creation date.

askmike 8y ago
Google crawls the open web non stop. They definitely have this data. It's just not exposed to the (legal) team dealing with DMCA takedown requests.. EDIT: Or as to the example above: whatever application is responsible for picking the main article to show somewhere.
- codetrotter 8y ago
  They crawl the web non-stop yes, but how often an individual site is crawled varies. > Our crawl process is algorithmic: computer programs determine how often we should crawl each site. If you post new articles on your site throughout the day, you should be able to see them appear on Google News fairly quickly. > [...] > From the moment we discover a new article, we'll keep re-crawling it looking for changes. Since we noticed that most changes to articles occur just after they're published, we revisit articles most frequently in the first day after we've found them. After that, we visit them less often. https://support.google.com/news/publisher-center/answer/40392?hl=en https://support.google.com/news/publisher-center/answer/4039...
  - askmike 8y ago
    Fair point, if your content gets copied to unknown website X which Google hasn't crawled for days they might not be able to tell where the article appeared first.
IncRnd 8y ago
Have you figured out a way to verify what a page states as its creation date?
- PeterisP 8y ago
  Ignore the claimed creation date, and use the earliest date that you know for certain, i.e. the time you crawled it.
  - jacquesm 8y ago
    That only works if you happen to crawl the original first. Given that copies could be plentiful that's a losing game. Content creators signing their content with a private key would be a better idea.
    - icebraining 8y ago
      Content creators signing their content with a private key would be a better idea. How would that help? Wouldn't the copier simply sign with their own key? Maybe Google should run a timestamping service. It could be stateless: send an hash of the content, get it back timestamped and signed, then include it in the page before publishing it.
      - jacquesm 8y ago
        They could, but it would not be registered before the other one would be. - create signature - post signature, possibly publicly, (wow, maybe an actual use for a blockchain) - post content Anybody copying it could be proven to be a copyist.
        icebraining 8y ago
        Right, you'd need something like the blockchain - ie, a timestamping server - to validate the posting date. But then you don't need a signature, just a hash. The advantage of using Google's timestamping service is that it wouldn't need to store the hash/signature, unlike the blockchain.
dexen 8y ago
This could be solved with cryptographic timestamps [1]. The current generation of tooling is a bit raw - cli utility and Python library - but fully usable on the publisher's end. If the practice picked up any traction, surely more front-end tooling would be created. Example workflow: - write a blogpost - build a canonical representation based on a simple HTML or XML template; including canonical URL, title, content, authorship (emails, social media URLs, etc.), optionally related multimedia in a Zip archive, perhaps inlined via data URL - alternatively, re-purpose a common Web scrapping format, like the one used by Firefox, for canonical representation - submit the canonical representation for timestamping [2] - provide a <meta name="publication-timestamp" value="canonical representation url"> in the document as published, to facilitate automatic and manual verification - upon dispute, provide the canonical representation and verify in the relevant timestamping services [1] for example http://opentimestamps.org/ http://opentimestamps.org/ [2] from experience, the processing takes under 30 minutes with OpenTimeStamps
- gambler 8y ago
  I agree that we need cryptographic timestamps. Seems like this would be a good use for blockchain, but any implementation would be better than nothing, as long as it's not fully centralized. The proposed workflow seems overly complex, though. For authorship verification all you need is ability to submit a block of text and receive a signed timestamp. Input: text Output: timestamp + signature Verification input: text + timestamp + signature. The simpler it is, the more universally it will be used. It should not 20 minutes either. The crypto involved should be pretty trivial. Any metadata could be built into the text itself as comments or hidden fields. However, this will not verify the URL the data is coming from, which might be desirable. The second kind of service could be verifiable archival service. Input: URL. Output: page content after rendering + timestamp + signature. Verification input: text + timestamp + signature. The service stores: timestamp + signature. This could be used to verify that at time X URL Y contained text Z. It doesn't matter whether the text is dynamic and go change later, since you could keep original page content.
  - dexen 8y ago
    > It should not [take?] 20 minutes either. Fair enough; the tech I know and mentioned (OpenTimestamps) is tied indirectly to Bitcoin's blockchain confirmation speed, thus both slow-ish, and also probabilistic rather than guaranteed. On the other hand, it gives good assurance that the tool will be around for a long time, and able to survive any single service provider's failure. > The crypto involved should be pretty trivial. Depends on the requirements. The tech I know and use (OTS) is privacy-conscious and handles timestamping of documents without revealing their actual content. Instead you submit hash(hash(document) + random, private IV) [1], which provides for both anonymous use, and ability to keep the signed content private, as the signed hash cannot be directly tied to any plaintext. This isn't strictly necessary for publicly-posted web pages, but certainly is useful for private documents, security-sensitive documents, etc. All in all, it's a starting point. Another possible approach would be to standardize submission to, and verification through a service like https://archive.is/ https://archive.is/, but that puts all trust into one 3rd party. > Input: [plain]text Directly timestamping the plaintext fails badly if the timestamping service provider is dishonest and creates an earlier timestamp for the content in their name, only to turn around and accuse the author of stealing the content. Or - a slightly paranoid scenario, but probably warranted in the post-Snowden era - if a whistleblower tries to timestamp some materials before distribution, but the timestamping service provider's filters catch the document and instead alert the government. Ergo, I'd prefer a blind timestamp, with the ability to reveal at the discretion of the author or whistleblower, even at the cost of more complex workflow. [1] sadly this means the author needs to keep a file containing the IV and a bit of other metadata.

johnchristopher 8y ago

What did you and the client do then ?