5 ms·
Good find! Now if only there was a way to download them all at once...
by sturmen 8y ago
Good find! Now if only there was a way to download them all at once...
- sosuke 8y agoThat is what a good coding project is for! If you've got Python experience, or just want to have fun with it, BeautifulSoup can do exactly what you want. There might be Firefox plugins as well.
- luke0016 8y agowget?
- acomjean 8y agowget with wildcards is what we use to get biology data. Those are ftp sites, so some experimentation may be necessary to make it work with http...
- luke0016 8y agoWhat? No, it works fine with http out of the box. From the man page: GNU Wget is a free utility for non-interactive download of files from the Web. It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies.
- ejstronge 8y agoI think the GP's uncertainty relates to download URL wildcards with HTTP. Unlike FTP, I don't believe that HTTP servers have to support your request to list all files in a directory.
- luke0016 8y agoThankfully the particular website in question already spits out a file listing for you.
- codemati 8y agoI wrote my first Python script to do it: https://github.com/cdtinney/lunar-scrape https://github.com/cdtinney/lunar-scrape Full disclosure: it's slow and there's no progress feedback.