15 ms·
On the topic of Geocities shutting down, I found an option in wget which ignores existing files - because Yahoo has a 503 error if you try to spider too many pa
by zandorg 17y ago
On the topic of Geocities shutting down, I found an option in wget which ignores existing files - because Yahoo has a 503 error if you try to spider too many pages.
It's -nc which means 'no clobber' - which is the least helpful phrase considering you could say 'ignore existing'...
When you get a 503, wait about an hour, and then run wget until it fails, then wait an hour, etc, until you've got all 10mb of your webspace.
Also -np to only spider below your page on Geocities.
- thwarted 17y ago"no clobber" matches the shell option of the same name.