4 ms·
Analyzing the text is the problem. Not extracting. Are there any good open source libs out there?
by dexcs 8y ago
Analyzing the text is the problem. Not extracting. Are there any good open source libs out there?
- msravi 8y agoSure. But the tool posted here doesn't do that. It merely extracts text, and the "analysis" is a couple of regexes that are tailor-made for that particular pdf. Awk can do that much and a lot more. If you want to extract tables from a pdf, there's Tabula[1], but it isn't automated to run over the whole pdf - you've to do a manual rectangular selection around the table you want to extract. 1. https://github.com/tabulapdf/tabula https://github.com/tabulapdf/tabula
- deleted 8y ago[deleted]
- deleted 8y ago[deleted]
- mirimir 8y agoIndeed. Many years ago, I "ran SQL" on a couple decades of Usenet newsgroup data. Extraction and manipulation involved a bunch of grep, sed, tr and awk (and millions of tmp files). But, as with PDFs of utility bills, it was very specific regex.
- kwadhwa 8y agoHey, Kshitij from Rockset here. With Rockset you can avoid ETL when it comes to extracting and manipulating the data. Also, the main value here is that you can join this data with other data sets that are in JSON, CSV, XLS or Parquet formats using SQL to help in analysis.
- mirimir 8y agoMaybe you could add modules for extracting and manipulating data from popular sources. Such as the most popular social media. Also Amazon, Craigslist, Ebay, etc. And the main search engines. There are many people who want usable data from such sources. And your service wouldn't be doing any scraping, so you'd probably be OK legally. But IANAL, so do check.
- danzig13 8y agoI’ve been impressed with Camelot for PDF tables