5 ms·

The author of the tweet also supplies a tool for cleaning the metadata in pdfs. https://gist.github.com/sneakers-the-rat/172e8679b824a3871decd262ed3f59c6 https

The author of the tweet also supplies a tool for cleaning the metadata in pdfs. https://gist.github.com/sneakers-the-rat/172e8679b824a3871decd262ed3f59c6 https://gist.github.com/sneakers-the-rat/172e8679b824a3871de...

hdjjhhvvhga 5y ago
This is a very basic tool. It may work against this particular scheme by Elsevier if it's not too sophisticated. A few years ago I was working for another content provider to mark the PDFs, my job was to make sure the watermarks are everywhere and that removing them is difficult while being practically transparent to the user. We also employed steganography etc. so that rendering the PDFs as images and combining them again into PDF would preserve the watermark. We were very clear to the user the watermarks are there and, as far as I can tell, there was no single copy protected in this way uploaded to the public Internet.
- sam_lowry_ 5y ago
  I wonder how you coped with the ethical side of such a job.
  - hdjjhhvvhga 5y ago
    Here's the story. A middle-size publisher contacted me about the problem they had: someone was distributing their PDFs online using various services, but mainly Scribd. At that time (it was maybe 2012?) it worked like this: the attacker released a document, notified his group, and they started downloading. The publisher immediately filed the DMCA copyright infringement notification and it was handled within 3-4 days - more than enough for people to download the "release". It was going on for a long time and Scribd refused to remove the account of the attacker.[0] After a couple of months they finally gave in and deleted it but of course a new account was immediately created. It was not a mouse-and-cat game, but more like mouse-and-turtle. So when they asked my help I told them, you don't have a copyright problem, you have a SEO problem - since people looking for their publications could find them easily online with the stolen copies sometimes appearing higher on SERP than their own. They said they would address that and they somehow came to terms with the fact that the past documents are lost, but they would like to avoid this situation in the future. I checked some of their publications. They were moderately priced and the content was quite interesting. Many of their customers were very supportive - actually they were sending reports when they noticed the pirated copies. And they weren't saying "Why should I pay for your books when I can get them online?" but rather "Please take care of protecting your content since we want you to survive and publish more books". So, to answer your question, I was very happy to work for them, and I would do it again. [0] I was told they changed their approach later and started to collaborate with publishers.
    - barry-cotter 5y ago
      > someone was distributing their PDFs online using various services, but mainly Scribd Ah, Scribd, the scummiest company YC has ever had any connection with.
      - onphonenow 5y ago
        So many dark patterns on that site
        SpelingBeeChamp 5y ago
        What are they tricking users into doing?
  - afandian 5y ago
    As a customer I want a thriving, commercially viable, decentralised publishing industry. I don't want DRM encumbered formats, or a single vendor, or a point of failure that prevents me from reading books I've purchased. If my email address embedded in the PDF enables that then I think that's reasonably balanced ethics arithmetic. (edit - I consider public-funded scholarly research to be a different matter to private purchases of commercial books such as fiction or trade textbooks)
- amself 5y ago
  I would just render each page of such a PDF in two colors (black or white without shades) and turn the images into a new PDF to upload. If that doesn’t make the steganography fail, maybe add a lot of static? Or censor illustrations anyway since many journals provide them independently of the text.
  - hdjjhhvvhga 5y ago
    It wouldn't work since we were allowed to manipulate certain aspects of text, too, to a certain extent, in a way that looked pretty much statistically random. That is, by comparing several copies of the same file, the bits were very much different, and the differences were distributed over the whole document. One of the main aims was to make sure the watermark gets preserved over various transformations, and we automatically tested each new document to make sure it works.
    - deleted 5y ago
      [deleted]
    - berkes 5y ago
      You could even just move a few words around, depending on the language. Depending on the language, you could even just move a few words around. Do that randomly, combine the products, and you might get enough entropy to create unique fingerprints for each download. Randomly do that, combine the products, and you might get enough entropy to create unique fingerprints for each download. (This silly example can create 4 unique fingerprints)
      - Wowfunhappy 5y ago
        When I write, I put a great deal of thought into how to arrange sentences for maximum clarity or effectiveness. I would not appreciate an eBook service messing with that, even if the meaning was unchanged. In the most extreme case, imagine if this was a book of poetry.
        viraptor 5y ago
- aqme28 5y ago
  I don’t think there is any scheme that works against all steganography, outside of maybe rewriting the whole document “in your own words.”
  - afandian 5y ago
    I'm curious what a legal case would look like around steganography. "Trust that our system says that this string is embedded". Or would the prosecution be obliged to divulge the algorithm.
    - hdjjhhvvhga 5y ago
      I believe it would be enough to demonstrate the functioning of the system in action - there was a crude UI that you could use to extract the string from the input document, so it would be easy to demonstrate the PDFs uploaded by the attacker to a given service do in fact contain that strings. But in this particular point it wasn't even necessary. After a couple of months it turned out that the person who had been uploading the unprotected versions made a mistake and was located as a 20-something living with his parents in a small house in the East Coast. It was enough to notify them and the malicious activity stopped. The company wasn't interested in extracting every penny from the the kid (or his poor parents), they just wanted him to stop, and one letter from a lawyer was enough. If they wanted to go full steam, they would have involved the police and I'm sure they would have found quite a lot of incriminating evidence on his computer, but they were clear ruining someone else's life was not their aim.
madars 5y ago
The gold standard for academic watermark removal used to be (and maybe still is?) https://github.com/kanzure/pdfparanoia https://github.com/kanzure/pdfparanoia . Not sure if it edits Elsevier's metadata though.
NKosmatos 5y ago
The tweet author also mentions these tools than can help in cleaning PDFs: exiftool: https://exiftool.org https://exiftool.org qpdf: https://qpdf.sourceforge.io https://qpdf.sourceforge.io dangerzone (GUI, render PDF as images, then re-OCR everything): https://dangerzone.rocks https://dangerzone.rocks mat2 (render PDF as images, don't OCR): https://0xacab.org/jvoisin/mat2 https://0xacab.org/jvoisin/mat2

hdjjhhvvhga 5y ago

This is a very basic tool. It may work against this particular scheme by Elsevier if it's not too sophisticated. A few years ago I was working for another content provider to mark the PDFs, my job was to make sure the watermarks are everywhere and that removing them is difficult while being practically transparent to the user. We also employed steganography etc. so that rendering the PDFs as images and combining them again into PDF would preserve the watermark. We were very clear to the user the watermarks are there and, as far as I can tell, there was no single copy protected in this way uploaded to the public Internet.

sam_lowry_ 5y ago
I wonder how you coped with the ethical side of such a job.
- hdjjhhvvhga 5y ago
  Here's the story. A middle-size publisher contacted me about the problem they had: someone was distributing their PDFs online using various services, but mainly Scribd. At that time (it was maybe 2012?) it worked like this: the attacker released a document, notified his group, and they started downloading. The publisher immediately filed the DMCA copyright infringement notification and it was handled within 3-4 days - more than enough for people to download the "release". It was going on for a long time and Scribd refused to remove the account of the attacker.[0] After a couple of months they finally gave in and deleted it but of course a new account was immediately created. It was not a mouse-and-cat game, but more like mouse-and-turtle. So when they asked my help I told them, you don't have a copyright problem, you have a SEO problem - since people looking for their publications could find them easily online with the stolen copies sometimes appearing higher on SERP than their own. They said they would address that and they somehow came to terms with the fact that the past documents are lost, but they would like to avoid this situation in the future. I checked some of their publications. They were moderately priced and the content was quite interesting. Many of their customers were very supportive - actually they were sending reports when they noticed the pirated copies. And they weren't saying "Why should I pay for your books when I can get them online?" but rather "Please take care of protecting your content since we want you to survive and publish more books". So, to answer your question, I was very happy to work for them, and I would do it again. [0] I was told they changed their approach later and started to collaborate with publishers.
  - barry-cotter 5y ago
    > someone was distributing their PDFs online using various services, but mainly Scribd Ah, Scribd, the scummiest company YC has ever had any connection with.
    - onphonenow 5y ago
      So many dark patterns on that site
      - SpelingBeeChamp 5y ago
        What are they tricking users into doing?
- afandian 5y ago
  As a customer I want a thriving, commercially viable, decentralised publishing industry. I don't want DRM encumbered formats, or a single vendor, or a point of failure that prevents me from reading books I've purchased. If my email address embedded in the PDF enables that then I think that's reasonably balanced ethics arithmetic. (edit - I consider public-funded scholarly research to be a different matter to private purchases of commercial books such as fiction or trade textbooks)
amself 5y ago
I would just render each page of such a PDF in two colors (black or white without shades) and turn the images into a new PDF to upload. If that doesn’t make the steganography fail, maybe add a lot of static? Or censor illustrations anyway since many journals provide them independently of the text.
- hdjjhhvvhga 5y ago
  It wouldn't work since we were allowed to manipulate certain aspects of text, too, to a certain extent, in a way that looked pretty much statistically random. That is, by comparing several copies of the same file, the bits were very much different, and the differences were distributed over the whole document. One of the main aims was to make sure the watermark gets preserved over various transformations, and we automatically tested each new document to make sure it works.
  - deleted 5y ago
    [deleted]
  - berkes 5y ago
    You could even just move a few words around, depending on the language. Depending on the language, you could even just move a few words around. Do that randomly, combine the products, and you might get enough entropy to create unique fingerprints for each download. Randomly do that, combine the products, and you might get enough entropy to create unique fingerprints for each download. (This silly example can create 4 unique fingerprints)
    - Wowfunhappy 5y ago
      When I write, I put a great deal of thought into how to arrange sentences for maximum clarity or effectiveness. I would not appreciate an eBook service messing with that, even if the meaning was unchanged. In the most extreme case, imagine if this was a book of poetry.
      - viraptor 5y ago
aqme28 5y ago
I don’t think there is any scheme that works against all steganography, outside of maybe rewriting the whole document “in your own words.”
- afandian 5y ago
  I'm curious what a legal case would look like around steganography. "Trust that our system says that this string is embedded". Or would the prosecution be obliged to divulge the algorithm.
  - hdjjhhvvhga 5y ago
    I believe it would be enough to demonstrate the functioning of the system in action - there was a crude UI that you could use to extract the string from the input document, so it would be easy to demonstrate the PDFs uploaded by the attacker to a given service do in fact contain that strings. But in this particular point it wasn't even necessary. After a couple of months it turned out that the person who had been uploading the unprotected versions made a mistake and was located as a 20-something living with his parents in a small house in the East Coast. It was enough to notify them and the malicious activity stopped. The company wasn't interested in extracting every penny from the the kid (or his poor parents), they just wanted him to stop, and one letter from a lawyer was enough. If they wanted to go full steam, they would have involved the police and I'm sure they would have found quite a lot of incriminating evidence on his computer, but they were clear ruining someone else's life was not their aim.

madars 5y ago

The gold standard for academic watermark removal used to be (and maybe still is?) https://github.com/kanzure/pdfparanoia https://github.com/kanzure/pdfparanoia . Not sure if it edits Elsevier's metadata though.

NKosmatos 5y ago

The tweet author also mentions these tools than can help in cleaning PDFs: exiftool: https://exiftool.org https://exiftool.org qpdf: https://qpdf.sourceforge.io https://qpdf.sourceforge.io dangerzone (GUI, render PDF as images, then re-OCR everything): https://dangerzone.rocks https://dangerzone.rocks mat2 (render PDF as images, don't OCR): https://0xacab.org/jvoisin/mat2 https://0xacab.org/jvoisin/mat2