3 ms·
Curl is really handy if you want to write a quick crawler for some globs of urls. For example if you want to download a gallery of images with some fixed forma
by zawerf 7y ago
Curl is really handy if you want to write a quick crawler for some globs of urls.
For example if you want to download a gallery of images with some fixed format you can do
curl "http://example.com/imgs/[01-99].jpg" -o "#1.jpg"
And it will fetch them and name them with the correct zero padding.
Or if you want to download the first 10 pages of hn:
curl https://news.ycombinator.com/news?p=[1-10] -o "hn_page_#1.html"
It can glob pretty complicated patterns! See this for syntax: https://ec.haxx.se/cmdline-globbing.html https://ec.haxx.se/cmdline-globbing.html
- hk__2 7y agoNice, I didn’t know `curl` had globbing support. I usually do that using shell globing, as in: wget http://example.com/imgs/{01..99}.jpg