3 ms·
I use Mnemosyne quite a bit too. I also have enjoyed writing scrapers for data I wanted but wasn't being provided in a downloadable format - most commonly stuf
by Ixiaus 16y ago
I use Mnemosyne quite a bit too.
I also have enjoyed writing scrapers for data I wanted but wasn't being provided in a downloadable format - most commonly stuff on forums.
I wrote a scraper that scraped the entire textfiles.org website for their texts (mostly for fun), I wrote a scraper that logged in as me into my favorite phpBB forum and scraped every post in every forum in every thread - the script would create a directory for the forum, a directory within that named by a number to denote which page it was on, a directory within that named after the thread, a directory within that named by a number to denote which page it was on, and a text file containing only the content div (to retain the formatting of the original post).
I used Python and Mechanize for most of this - the phpbb forum scraper was the most fun and rewarding.
I've also written page scrapers for websites that have things like "1324 of the best side boobs!" I didn't want to Save As for all of them! So I wrote a script that downloaded them for me (this one was easy though compared to the phpbb script).