4 ms·

So the tool is being used to extract text and then a regex is used to extract the relevant fields/values. Seems that pdftotext[1][2] with awk can do the job on

• dexcs 8y ago
  Analyzing the text is the problem. Not extracting. Are there any good open source libs out there?
  • msravi 8y ago
    Sure. But the tool posted here doesn't do that. It merely extracts text, and the "analysis" is a couple of regexes that are tailor-made for that particular pdf. Awk can do that much and a lot more. If you want to extract tables from a pdf, there's Tabula[1], but it isn't automated to run over the whole pdf - you've to do a manual rectangular selection around the table you want to extract. 1. https://github.com/tabulapdf/tabula https://github.com/tabulapdf/tabula
    • deleted 8y ago
      [deleted]
    • deleted 8y ago
      [deleted]
    • mirimir 8y ago
      Indeed. Many years ago, I "ran SQL" on a couple decades of Usenet newsgroup data. Extraction and manipulation involved a bunch of grep, sed, tr and awk (and millions of tmp files). But, as with PDFs of utility bills, it was very specific regex.
      • kwadhwa 8y ago
        Hey, Kshitij from Rockset here. With Rockset you can avoid ETL when it comes to extracting and manipulating the data. Also, the main value here is that you can join this data with other data sets that are in JSON, CSV, XLS or Parquet formats using SQL to help in analysis.
        • mirimir 8y ago
          Maybe you could add modules for extracting and manipulating data from popular sources. Such as the most popular social media. Also Amazon, Craigslist, Ebay, etc. And the main search engines. There are many people who want usable data from such sources. And your service wouldn't be doing any scraping, so you'd probably be OK legally. But IANAL, so do check.
  • danzig13 8y ago
    I’ve been impressed with Camelot for PDF tables