3 ms·
Does this count as a make hack? I have (or had, can't seem to find it right now) a script to generate a makefile from a list of URLs of files to be downloaded.
by js2 10y ago
Does this count as a make hack? I have (or had, can't seem to find it right now) a script to generate a makefile from a list of URLs of files to be downloaded. In this case, it was for when I needed to download hundreds or thousands of files. The makefile just called curl to do the downloading. But the beauty of using make was that I could easily control the download concurrency with -j, and of course that it would skip already downloaded files. I think I may have even made it properly resume partial downloads by checking for the temporary target file and using its size to send the right range request.
Anyway, it's been a few years, but I thought it was pretty clever at the time.
- aidos 10y agoI'm not a guru, but that sounds like the sort of thing wget is built to solve. I'm also not a make guru - that sounds like a nifty use.
- dahart 10y agoYeah, I think it counts, though make was designed for general purpose jobs and not just compilation, so it gets a lot of credit for being a solid and general tool. Being able to resume jobs is awesome, and free concurrency is too. I've done the same thing and used make for batch downloads and batch image resizing and batch frame rendering, etc. etc. I did it enough times that I generalized my makefile so I could easily schedule resumeable parallel command line batches. I called it "mbatch" and was getting ready to publish it... but then I discovered gnu parallel. :P
- oso2k 10y agoYou have this posted anywhere? I'd be interested in studying it.