3 ms·
Warning: Self-Advertisment! http://www.joyofdata.de/blog/using-linux-shell-web-scraping/ http://www.joyofdata.de/blog/using-linux-shell-web-scraping/ Okay, it
by joyofdata 13y ago
Warning: Self-Advertisment!
http://www.joyofdata.de/blog/using-linux-shell-web-scraping/ http://www.joyofdata.de/blog/using-linux-shell-web-scraping/
Okay, it's not as bold has using Headers saying "Hire Me" but I would like to emphasize that sometimes even complex tasks can be super-easy when you use the right tools. And a combination of Linux shell tools makes this task really very straightforward (literally).
- jmduke 13y agoThis is really cool -- particularly hxselect -- but I don't see how it's particularly simpler than using Python/requests. How would you handle following links and deduping visited URLs via piping?
- joyofdata 13y agoThanks - well recursive downloading of a web-page using wget is no big deal at all. And regarding particularly simpler - it is particularly shorter and pretty clear what's going on.
- sdoering 13y agoThanks for the self-advertisement. That way, I was introduced to some fine reading on your blog. Greetings from Hamburg.
- joyofdata 13y agoThanks for the kind words - you're most welcome