111 ms·

Borb – A Python library to read, write, and edit PDF files

262 points by dsr12 5y ago

pixelmonkey 5y ago
The README in the GitHub repo for borb is a bit of a better explainer than this landing page (especially for Python programmers). https://github.com/jorisschellekens/borb/blob/master/README.md https://github.com/jorisschellekens/borb/blob/master/README....
- avnigo 5y ago
  And the example repository shows off its capabilities better too: https://github.com/jorisschellekens/borb-examples https://github.com/jorisschellekens/borb-examples
  - mrweasel 5y ago
    The examples are pretty useful, seeing as there apparently isn't any documentation yet. It a really amazing project. One of those that makes you go: "Wait we didn't have this before?"
alephu5 5y ago
Amazing, I've been yearning for something like this for years but have always been told it's impossible. Can't wait to try it
- eurasiantiger 5y ago
  It is literally impossible for all PDFs, since some of them may not have any kind of semantic structure and consist of a set of graph bitmaps laid out at specific coordinates to make up blocks of text.
  - anonymouse008 5y ago
    The best PDF bug is when the linker between the Adobe character value and the font/language is broken and you get random Unicode like values with no way to connect the two. Infuriating.
    - dunham 5y ago
      Some PDF files intentionally include a bad character mapping table (and reorder the font) as a form of DRM.
      - mkl 5y ago
        It's not very effective against anyone determined though. You can OCR easily. You can also rebuild the character mapping from the shapes of the glyphs, and in most languages there are few enough that you can even do it by hand.
        mcswell 5y ago
        You can OCR if you're using a Latin script with few if any accent marks. Depending on your OCR engine, I suppose you could do Cyrillic too. Other scripts, not so much. (And yes, I'm a computational linguist, so we deal with non-Roman scripts all the time, particularly Arabic script. But I suppose that's not a problem for most people here :-).) There might be some Latin script fonts that cause problems, but I haven't looked into that very much--I do recall we had problems with an italic font.
        dunham 5y ago
        When I came across this I already had a pristine copy of the font, so I just compared the program for each character to determine the mapping. (I was automating the decoding.) I agree that there is little to no security there. But the point that I was not so clearly trying to make was that sometimes the messed up encoding is intentional and not a bug.
  - deleted 5y ago
    [deleted]
wodenokoto 5y ago
Some books starts counting pages after the table of contents and so the pdf page number and the book page number are not in sync. I’ve seen some PDFs have the first few pages counted in Roman numerals and then “normal” numbers for the main content. How do you edit an existing pdf to do that?
- shakna 5y ago
  Page numbers are part of the PageInfo object within a PDF, but it looks like currently this is generated naively [0], so I don't think multiple numbering schemes are currently supported by this library. [0] https://github.com/jorisschellekens/borb/blob/master/borb/pdf/page/page_info.py#L67 https://github.com/jorisschellekens/borb/blob/master/borb/pd...
chrismorgan 5y ago
I have a PDF of a hymn book that I want to convert to 2-up so I can use it that way on my reMarkable which only supports single-page display and not two-page spreads; but I also want all the internal hyperlinks (to and from a table of contents) to keep working. I haven’t found any software that seems capable of doing this (though I’ve only looked at FOSS; wouldn’t surprise me if something Adobe could do it). The closest I seem to have found is qpdf which might be able to do it with some programming effort. Is that sort of thing going to be in scope for this library’s editing capabilities? (“Editing PDFs” is such a broad, open-ended thing.)
Syzygies 5y ago
I am a math professor with a scanned exam grading workflow that I hacked together as Bash scripts using various open source command line tools. I feed all the exams through a sheet-fed scanner, decode bar codes to identify problems and students, add radio buttons for entering and tracking scores (0-6 per problem), and create PDF "books" per problem for grading and annotating. Having grad students help grade paper is a consistency nightmare: It's look once, never look back. Instead, after each of several provisional passes I recreate the PDF "book" for that problem, with a chapter for each score, and students randomized within each chapter. In the same spirit as "checking your work lets you work three times faster" this is actually both more consistent and faster that a single pass over paper. Almost all of my attention is on the math, which I'm good at, rather than locating problems and finding again the ones I know I misgraded, which I'm not good at. Then each student's exam needs to be extracted from these problem PDFs, scores recorded, and annotations frozen. There are cloud services for grading. They're hopelessly primitive, with cloud lag. Like a gamer, I used to reject wireless mice because of the lag. I reject these services. I can grade everything myself faster than using a team of grad students, with the right local tools. The PDF format is a morass. My hats off to anyone who will work with it. There are many evolutionary layers and no formal specification or verification; one tests a PDF by seeing if most programs accept it. It's time for me to rewrite my grading system in a modern scripting language, so others could use it. I prefer Ruby, but that's mainly to stave off boredom when I'm not using Haskell. I can use Python. This would permit a more robust workflow, such as adding late exams in mid-grading without losing grading in progress. I can't find documentation for Borb, to check off the list of features I'd need. I suspect from this being a one-person project that I might need to continue to patch together external tools.
- jhgb 5y ago
  > Having grad students help grade paper is a consistency nightmare: It's look once, never look back. Captcha-ize them, with several of them grading the same result, and with checking their responses against each other?
- Tijdreiziger 5y ago
  Perhaps you might be interested in the Zesje project: https://gitlab.kwant-project.org/zesje/zesje https://gitlab.kwant-project.org/zesje/zesje
jl6 5y ago
Always good to see more open source tools for the PDF ecosystem. I couldn’t see any support for PDF/A (the good version of PDF) in borb though.
nickspain 5y ago
This is awesome! I've been looking for something that could be a link between PDFs and Instapaper[0] for a while. This looks like it'll be perfect to build such a tool with. [0] https://www.instapaper.com https://www.instapaper.com
einpoklum 5y ago
So, are there decent lower-level (e.g. C++ or even C) libraries for doing this, which this library wraps? Or does this actually do the nitty-gritty PDF innards itself? As for myself, I've not had to automate work on PDFs, luckily; for manual manipulation and annotation I've found Xournal++ sort of useful (https://xournalpp.github.io/ https://xournalpp.github.io/). Inkscape can also be used with some questionable PDFs.
- hansvm 5y ago
  > or does this actually do the nitty-gritty PDF innards itself? > with borb, a pure python library
spapas82 5y ago
Haven't tested this lib, however be careful before including it in your project because of its license (it is dual licensed agpl/commercial). This means that you can use it only if your project is GPL or else you need a commercial license. On the other hand, the reportlab pdf generation library (which is what I actually use) offers a permissive language in its open source version (and a commercial reportlab plus version), so it can be included in all kinds of projects.
- mkl 5y ago
  Confusingly, while the README says AGPL/commercial, the LICENSE file (https://github.com/jorisschellekens/borb/blob/master/LICENSE https://github.com/jorisschellekens/borb/blob/master/LICENSE) says it's GPL.
  - mdaniel 5y ago
    I'd guess it's just a copy-pasta error, because all the source files contain the AGPL header text: https://github.com/jorisschellekens/borb/blob/v2.0.9.1/borb/__init__.py#L7 https://github.com/jorisschellekens/borb/blob/v2.0.9.1/borb/... and https://github.com/jorisschellekens/borb/blob/v2.0.9.1/borb/pdf/__init__.py#L7 https://github.com/jorisschellekens/borb/blob/v2.0.9.1/borb/... for example But the plot thickens! It seems the top-level LICENSE file was actually changed 13 days ago _away_ from AGPL https://github.com/jorisschellekens/borb/blame/master/LICENSE https://github.com/jorisschellekens/borb/blame/master/LICENS... So, yeah, confusingly for sure
  - sigg3 5y ago
    An AMBIGUOUS LICENSE situation is a red flag. Was the issue raised with the author?
- xjlin0 5y ago
  ReportLab is indeed a great library. Another great one is WeasyPrint. How are these compared with Borb?
cycomanic 5y ago
So how does this compare to the python bindings of mupdf? Which IMO is the most featureful module to manipulate PDFs in python (I'm a bit buffled by all the comments that something like this didn't exist before).
- jacob019 5y ago
  Borb is pure python. MuPDF is C with bindings. Reportlab is pure python. I use reportlab combined with PyPDF2 and pdf-redactor. It would be nice to see a comparison with the existing tools.
rstuart4133 5y ago
There are a few PDF Python libraries and open source programs out there, but as far as I can tell all lack one feature: signing. If anyone knows of a open source tool kit or library that can sign, I'd be most appreciative.
sneak 5y ago
I was going to be upset if the project logo were not a fat birb. I was not disappointed. :)

262 points by dsr12 5y ago

The README in the GitHub repo for borb is a bit of a better explainer than this landing page (especially for Python programmers). https://github.com/jorisschellekens/borb/blob/master/README.md https://github.com/jorisschellekens/borb/blob/master/README....

avnigo 5y ago
And the example repository shows off its capabilities better too: https://github.com/jorisschellekens/borb-examples https://github.com/jorisschellekens/borb-examples
- mrweasel 5y ago
  The examples are pretty useful, seeing as there apparently isn't any documentation yet. It a really amazing project. One of those that makes you go: "Wait we didn't have this before?"

alephu5 5y ago

Amazing, I've been yearning for something like this for years but have always been told it's impossible. Can't wait to try it

eurasiantiger 5y ago
It is literally impossible for all PDFs, since some of them may not have any kind of semantic structure and consist of a set of graph bitmaps laid out at specific coordinates to make up blocks of text.
- anonymouse008 5y ago
  The best PDF bug is when the linker between the Adobe character value and the font/language is broken and you get random Unicode like values with no way to connect the two. Infuriating.
  - dunham 5y ago
    Some PDF files intentionally include a bad character mapping table (and reorder the font) as a form of DRM.
    - mkl 5y ago
      It's not very effective against anyone determined though. You can OCR easily. You can also rebuild the character mapping from the shapes of the glyphs, and in most languages there are few enough that you can even do it by hand.
      - mcswell 5y ago
        You can OCR if you're using a Latin script with few if any accent marks. Depending on your OCR engine, I suppose you could do Cyrillic too. Other scripts, not so much. (And yes, I'm a computational linguist, so we deal with non-Roman scripts all the time, particularly Arabic script. But I suppose that's not a problem for most people here :-).) There might be some Latin script fonts that cause problems, but I haven't looked into that very much--I do recall we had problems with an italic font.
      - dunham 5y ago
        When I came across this I already had a pristine copy of the font, so I just compared the program for each character to determine the mapping. (I was automating the decoding.) I agree that there is little to no security there. But the point that I was not so clearly trying to make was that sometimes the messed up encoding is intentional and not a bug.
- deleted 5y ago
  [deleted]

wodenokoto 5y ago

Some books starts counting pages after the table of contents and so the pdf page number and the book page number are not in sync. I’ve seen some PDFs have the first few pages counted in Roman numerals and then “normal” numbers for the main content. How do you edit an existing pdf to do that?

shakna 5y ago
Page numbers are part of the PageInfo object within a PDF, but it looks like currently this is generated naively [0], so I don't think multiple numbering schemes are currently supported by this library. [0] https://github.com/jorisschellekens/borb/blob/master/borb/pdf/page/page_info.py#L67 https://github.com/jorisschellekens/borb/blob/master/borb/pd...

chrismorgan 5y ago

I have a PDF of a hymn book that I want to convert to 2-up so I can use it that way on my reMarkable which only supports single-page display and not two-page spreads; but I also want all the internal hyperlinks (to and from a table of contents) to keep working. I haven’t found any software that seems capable of doing this (though I’ve only looked at FOSS; wouldn’t surprise me if something Adobe could do it). The closest I seem to have found is qpdf which might be able to do it with some programming effort. Is that sort of thing going to be in scope for this library’s editing capabilities? (“Editing PDFs” is such a broad, open-ended thing.)

Syzygies 5y ago

I am a math professor with a scanned exam grading workflow that I hacked together as Bash scripts using various open source command line tools. I feed all the exams through a sheet-fed scanner, decode bar codes to identify problems and students, add radio buttons for entering and tracking scores (0-6 per problem), and create PDF "books" per problem for grading and annotating. Having grad students help grade paper is a consistency nightmare: It's look once, never look back. Instead, after each of several provisional passes I recreate the PDF "book" for that problem, with a chapter for each score, and students randomized within each chapter. In the same spirit as "checking your work lets you work three times faster" this is actually both more consistent and faster that a single pass over paper. Almost all of my attention is on the math, which I'm good at, rather than locating problems and finding again the ones I know I misgraded, which I'm not good at. Then each student's exam needs to be extracted from these problem PDFs, scores recorded, and annotations frozen. There are cloud services for grading. They're hopelessly primitive, with cloud lag. Like a gamer, I used to reject wireless mice because of the lag. I reject these services. I can grade everything myself faster than using a team of grad students, with the right local tools. The PDF format is a morass. My hats off to anyone who will work with it. There are many evolutionary layers and no formal specification or verification; one tests a PDF by seeing if most programs accept it. It's time for me to rewrite my grading system in a modern scripting language, so others could use it. I prefer Ruby, but that's mainly to stave off boredom when I'm not using Haskell. I can use Python. This would permit a more robust workflow, such as adding late exams in mid-grading without losing grading in progress. I can't find documentation for Borb, to check off the list of features I'd need. I suspect from this being a one-person project that I might need to continue to patch together external tools.

jhgb 5y ago
> Having grad students help grade paper is a consistency nightmare: It's look once, never look back. Captcha-ize them, with several of them grading the same result, and with checking their responses against each other?
Tijdreiziger 5y ago
Perhaps you might be interested in the Zesje project: https://gitlab.kwant-project.org/zesje/zesje https://gitlab.kwant-project.org/zesje/zesje

jl6 5y ago

Always good to see more open source tools for the PDF ecosystem. I couldn’t see any support for PDF/A (the good version of PDF) in borb though.

nickspain 5y ago

This is awesome! I've been looking for something that could be a link between PDFs and Instapaper[0] for a while. This looks like it'll be perfect to build such a tool with. [0] https://www.instapaper.com https://www.instapaper.com

einpoklum 5y ago

So, are there decent lower-level (e.g. C++ or even C) libraries for doing this, which this library wraps? Or does this actually do the nitty-gritty PDF innards itself? As for myself, I've not had to automate work on PDFs, luckily; for manual manipulation and annotation I've found Xournal++ sort of useful (https://xournalpp.github.io/ https://xournalpp.github.io/). Inkscape can also be used with some questionable PDFs.

hansvm 5y ago
> or does this actually do the nitty-gritty PDF innards itself? > with borb, a pure python library

spapas82 5y ago

Haven't tested this lib, however be careful before including it in your project because of its license (it is dual licensed agpl/commercial). This means that you can use it only if your project is GPL or else you need a commercial license. On the other hand, the reportlab pdf generation library (which is what I actually use) offers a permissive language in its open source version (and a commercial reportlab plus version), so it can be included in all kinds of projects.

mkl 5y ago
Confusingly, while the README says AGPL/commercial, the LICENSE file (https://github.com/jorisschellekens/borb/blob/master/LICENSE https://github.com/jorisschellekens/borb/blob/master/LICENSE) says it's GPL.
- mdaniel 5y ago
  I'd guess it's just a copy-pasta error, because all the source files contain the AGPL header text: https://github.com/jorisschellekens/borb/blob/v2.0.9.1/borb/__init__.py#L7 https://github.com/jorisschellekens/borb/blob/v2.0.9.1/borb/... and https://github.com/jorisschellekens/borb/blob/v2.0.9.1/borb/pdf/__init__.py#L7 https://github.com/jorisschellekens/borb/blob/v2.0.9.1/borb/... for example But the plot thickens! It seems the top-level LICENSE file was actually changed 13 days ago _away_ from AGPL https://github.com/jorisschellekens/borb/blame/master/LICENSE https://github.com/jorisschellekens/borb/blame/master/LICENS... So, yeah, confusingly for sure
- sigg3 5y ago
  An AMBIGUOUS LICENSE situation is a red flag. Was the issue raised with the author?
xjlin0 5y ago
ReportLab is indeed a great library. Another great one is WeasyPrint. How are these compared with Borb?

cycomanic 5y ago

So how does this compare to the python bindings of mupdf? Which IMO is the most featureful module to manipulate PDFs in python (I'm a bit buffled by all the comments that something like this didn't exist before).

jacob019 5y ago
Borb is pure python. MuPDF is C with bindings. Reportlab is pure python. I use reportlab combined with PyPDF2 and pdf-redactor. It would be nice to see a comparison with the existing tools.

rstuart4133 5y ago

There are a few PDF Python libraries and open source programs out there, but as far as I can tell all lack one feature: signing. If anyone knows of a open source tool kit or library that can sign, I'd be most appreciative.

sneak 5y ago

I was going to be upset if the project logo were not a fat birb. I was not disappointed. :)