4 ms·
How is it updated hourly? Did you write your own crawler to pull the data?
by sillysaurus 13y ago
How is it updated hourly? Did you write your own crawler to pull the data?
- sloria 13y agoMore info here: https://github.com/sloria/python-subreddit-stats https://github.com/sloria/python-subreddit-stats. Not much to it--the script uses BeautifulSoup to scrape user count, and an Openshift app runs the script hourly as a cronjob.
- sillysaurus 13y agoI don't see the scraper (the BeautifulSoup script) at that repo. Did I miss it or is it not committed?
- sloria 13y agoLink was in the README: https://github.com/sloria/datasources/blob/master/scripts/rusers.py https://github.com/sloria/datasources/blob/master/scripts/ru...