6 ms·
What does it use to scrape? Is it just an xml/html parser, or does it run a headless browser?
by Fudgel 7y ago
What does it use to scrape? Is it just an xml/html parser, or does it run a headless browser?
- scrape_it 7y agoIt uses a sort of a DSL that I came up with, and removes a lot of boilerplate. Headless yes. You write WSL (Web Scraping Language) to scrape 2 pages of HN: GOTO news.ycombinator.com | EXTRACT {.morelink, 2} {news: .storylink} (you may certainly remove that ' , 2 ' business based on the moral system you subscribe to)