2 ms·
Where is a HN mod when you need one? The original title of the OP is much more accurate and much more interesting: "Probabalistic Scraping of Plain Text Tables
by danso 13y ago
Where is a HN mod when you need one? The original title of the OP is much more accurate and much more interesting:
"Probabalistic Scraping of Plain Text Tables"
vs the submitted title, "Accurate, efficient and maintainable scrapers"
As someone who's scraped a lot of plain text government data, I've given up on finding a flexible machine-learning way and just hand-coded for specific usecases (because government software changes so rarely). However, the OP presents an interesting strategy (with math that I haven't quite mastered, unfortunately)
- tlarkworthy 13y agoits not so hard, if you understand how to force boolean algebra into intger form, like: x + y >= 1, where x={0,1} is a boolean OR between x, y, then it is easy to expressed in PuLP like: x = variable("x", lowerbound=0,upperbound=1, cat='Integer') y = variable("y", lowerbound=0,upperbound=1, cat='Integer') problem += x + y >= 1 tada its really easy to write (once the math is defined, the modelling bit is the hard part, but it gets easier over time)