3 ms·
Show HN: Created Pickaxe a SQL like DSL for web scraping
- breeve 10y agoExamples that run are here https://github.com/bitsummation/pickaxe/tree/master/Examples https://github.com/bitsummation/pickaxe/tree/master/Examples
- scrollaway 10y agoInteresting. That looks like a lot of work. What are the main advantages compared to just a regular scraper framework like scrapy, which doesn't require learning a dsl?
- breeve 10y agoIt uses SQL syntax. If you know SQL and how CSS selectors work, there isn't much DSL to learn. In my experience there are lots of non programmers that understand SQL.
- MichaelBurge 10y agoI thought XQuery was standard for a lot of this?
- breeve 10y agoI think CSS selectors are easier to work with. Pickaxe has more features than just querying like downloading pages in parallel. Html also tends to not be well formed.