3 ms·

If I understand you right, you have a lot of different data types to scrape, so essentially you have a sub-program for each data type and when a page is downloa

by troels 12y ago

If I understand you right, you have a lot of different data types to scrape, so essentially you have a sub-program for each data type and when a page is downloaded, you let each of these have a go at the page and emit content if it finds any? Or did I completely miss the point?

boie0025 12y ago
Yeah, I think we're on the same page. I just hacked together a quick example at this gist: https://gist.github.com/boie0025/ae9697eed61cbf5342a6 https://gist.github.com/boie0025/ae9697eed61cbf5342a6
- troels 12y ago
  Thanks for the snippet - make sense.