3 ms·

> The issue with Python is missed value, not the value it can deliver. I'm very aware of the business value here! The limiting factor in these types of "messy

> The issue with Python is missed value, not the value it can deliver. I'm very aware of the business value here! The limiting factor in these types of "messy real-world data" problems is the developer time to get 100+ templates right on all the different variations encountered in the wild. I can iterate on each template extremely effectively in a Jupyter notebook REPL, and immediately rerun a sample of 100 statements for that bank in a few seconds. While the total corpus of statements I have access to is actually around a million, no one cares how quick processing them all are if the extraction isn't reliable enough!

aidos 4y ago
Exactly. The time spent developing a solution for your problem in pretty much any other language is going to cost magnitudes more than processing millions of documents in python. And that’s ignoring the ongoing maintenance where the hackability of teasing data out of PDFs in ipython is going to top any other system. I have a soft spot for the work you’re doing since I’ve spent a good portion of my life now extracting data from PDFs and can appreciate the joys of the process more than most.