5 ms·

I still couldn't find a tool for a difficult problem to solve. I have some magazines in PDF, with layouts in two columns, etc. I want them to be transformed int

by 101008 3y ago

I still couldn't find a tool for a difficult problem to solve. I have some magazines in PDF, with layouts in two columns, etc. I want them to be transformed into Markdown. I know, it should identify automatically the two columns, different layouts, etc. I am not desiring something perfect - I can fix if ther are some errors, but so far nothing has come with a good result.

layer8 3y ago
This can be arbitrarily difficult to do, depending on the PDF. This is generally called PDF reflowing. Another approach is to use column-aware OCR software.
qingcharles 3y ago
This is a hard problem. Cut the PDF down so it's only the pages of the article you want and then try feeding it through GPT Pro or Claude?
jftuga 3y ago
Have you tried this (for at least solving part of the problem)? https://github.com/pdfcpu/pdfcpu https://github.com/pdfcpu/pdfcpu
solardev 3y ago
I use Briss for this: https://github.com/mbaeuerle/Briss-2.0 https://github.com/mbaeuerle/Briss-2.0 It overlays all the pages on top of each other, you the human draws rectangles around the stacked columns in the easy GUI, and then it processes them into pages.
stavros 3y ago
Depending on how much you're willing to pay, the OpenAI GPT-Vision API can definitely do this extremely well.
- 101008 3y ago
  I am willing to pay for this, I only have around ~80 files, with 30 pages each (average). Is there a quick way to test this without wasting too much on the code part?
  - stavros 3y ago
    Yes, ChatGPT will do it if you upload photos of the page.
    - 101008 3y ago
      I just tried it with the API and it worked better than expected. Now I need to find to convert PDF to JPG with an API, and find the best prompt to ask GPT to only convert to markdown articles and not pages with columns, ads, etc. Thanks!
      - stavros 3y ago
        No problem! There are many Linux programs (some in this thread) to convert a pdf to images, and the prompt will hopefully not be too hard. I started making app to read our board game cards out loud (with voice) for our horror board game nights (https://boardguru.net https://boardguru.net) and GPT-4 could read cards that I couldn't make out!
      - slig 3y ago
        >Now I need to find to convert PDF to JPG with an API You can do that very easily with a locally installed ImageMagick. ChatGPT can help with the commands needed, but should be just one to convert a PDF to a number of JPGs and a small shell script to run on all your files.
jbverschoor 3y ago
The built-in apple preview application does exactly what you want.. It just looses bold etc.
pklee 3y ago
Have you considered marker. Does a very good job of turning PDF into markdown. - https://github.com/VikParuchuri/marker https://github.com/VikParuchuri/marker
rmbyrro 3y ago
I bet GPT vision will do that like cutting a piece of cake. It'll even do the OCR for you and organize the text nicely.
pseingatl 3y ago
There's this: https://www.cp.eng.chula.ac.th/~somchai/cut2col/ https://www.cp.eng.chula.ac.th/~somchai/cut2col/ Free and it works pretty well.

by 101008 3y ago

layer8 3y ago

This can be arbitrarily difficult to do, depending on the PDF. This is generally called PDF reflowing. Another approach is to use column-aware OCR software.

qingcharles 3y ago

This is a hard problem. Cut the PDF down so it's only the pages of the article you want and then try feeding it through GPT Pro or Claude?

jftuga 3y ago

Have you tried this (for at least solving part of the problem)? https://github.com/pdfcpu/pdfcpu https://github.com/pdfcpu/pdfcpu

solardev 3y ago

I use Briss for this: https://github.com/mbaeuerle/Briss-2.0 https://github.com/mbaeuerle/Briss-2.0 It overlays all the pages on top of each other, you the human draws rectangles around the stacked columns in the easy GUI, and then it processes them into pages.

stavros 3y ago

Depending on how much you're willing to pay, the OpenAI GPT-Vision API can definitely do this extremely well.

101008 3y ago
I am willing to pay for this, I only have around ~80 files, with 30 pages each (average). Is there a quick way to test this without wasting too much on the code part?
- stavros 3y ago
  Yes, ChatGPT will do it if you upload photos of the page.
  - 101008 3y ago
    I just tried it with the API and it worked better than expected. Now I need to find to convert PDF to JPG with an API, and find the best prompt to ask GPT to only convert to markdown articles and not pages with columns, ads, etc. Thanks!
    - stavros 3y ago
      No problem! There are many Linux programs (some in this thread) to convert a pdf to images, and the prompt will hopefully not be too hard. I started making app to read our board game cards out loud (with voice) for our horror board game nights (https://boardguru.net https://boardguru.net) and GPT-4 could read cards that I couldn't make out!
    - slig 3y ago
      >Now I need to find to convert PDF to JPG with an API You can do that very easily with a locally installed ImageMagick. ChatGPT can help with the commands needed, but should be just one to convert a PDF to a number of JPGs and a small shell script to run on all your files.

jbverschoor 3y ago

The built-in apple preview application does exactly what you want.. It just looses bold etc.

pklee 3y ago

Have you considered marker. Does a very good job of turning PDF into markdown. - https://github.com/VikParuchuri/marker https://github.com/VikParuchuri/marker

rmbyrro 3y ago

I bet GPT vision will do that like cutting a piece of cake. It'll even do the OCR for you and organize the text nicely.

pseingatl 3y ago

There's this: https://www.cp.eng.chula.ac.th/~somchai/cut2col/ https://www.cp.eng.chula.ac.th/~somchai/cut2col/ Free and it works pretty well.