Thanks for the link. I saw that. I was hoping to find something using webkit. I found dryscrape but wasn't sure how popular it is. Also I was trying to find out

Thanks for the link. I saw that. I was hoping to find something using webkit. I found dryscrape but wasn't sure how popular it is. Also I was trying to find out what the standard practice is. Do people build it to suit their needs so that they can optimize it for scale and performance? The scraping need seems generic enough to me to create a library or tool similar to dryscrape. The only problem is dryscrape is specifically for linux. I was hoping to to find some cross platform solution so that I can have it on my dev machine before moving it to the linux server.