As a PoC, I would be willing to "turn the web into data", i.e., produce one of the formats offered by these "services": CSV. I will use only standard UNIX util

As a PoC, I would be willing to "turn the web into data", i.e., produce one of the formats offered by these "services": CSV. I will use only standard UNIX utilities, no Python, etc. As such, you "own" the code. No SaaS. The result will be portable and run on any UNIX. I believe I can deliver in fewer words of code and that the result will be easier to modify when sites change. You pay nothing. Post your scraping "challenges" to HN. I enjoy turning web into data. Some people enjoy working with HTML, CSS, Javascript, etc. I prefer working with raw data. It is interesting to hear that some people are willing to pay to have the HTML, CSS, Javascript, etc. stripped out.