3 ms·
I've traditionally managed this with just 1 LoC: wget --mirror --page-requisites --convert-links localhost:9292 But this is probably more robust. EDIT: f
by JonnieCache 15y ago
I've traditionally managed this with just 1 LoC:
wget --mirror --page-requisites --convert-links localhost:9292
But this is probably more robust.
EDIT: for bonus points, cd into your wget-created mirror, fire up an http server with
python -m SimpleHTTPServer
Then check for any broken links with
wget --spider --page-requisites locahost:8000
wget is cool.
Ultimately though, correct use of cache headers and varnish or similar gives you the same effect automatically, if you're just looking for speed.
- jamesbritt 15y agoI don't know if this is clever or not, but it was practical for me: I wanted to have some basic sanity checks on all public (i.e. non-log-in) pages of a site, so I used wget to mirror it. What I was really after was the wget progress output so I could get a list of all the page URLs. I then used that to drive Watir to hit every page and a) check for any 404 or such errors and b) take a screen shot and save it off. I could then fire up an image browser and quickly see if any pages looked wonky ( missing or misaligned images images, crappy text layout, etc.). wget is very cool. :)