5 ms·
Noted! It may not be feasible for the first stage of the process though as the system does not actually know how many URLs are in the site. I'll definitely hav
by aarongough 17y ago
Noted! It may not be feasible for the first stage of the process though as the system does not actually know how many URLs are in the site.
I'll definitely have a look into that though.
- godDLL 17y agoYou might want to look at doing this in several passes: - Build a site-map, like parsers build a syntax tree. - Follow that to validate one page at a time.
- aarongough 17y agoThat is certainly a valid way of doing it and is the methodology that I have used in several search-engine projects in the past (like this: http://intrasitesearchsupport.com/ http://intrasitesearchsupport.com/) However in this case I am offloading the work of building the site-map to the WDG Validation service as I didn't want to have to obtain new servers to provide a free service. This means that I don't get the site-map until the WDG results come back...