3 ms·
I thought of doing this but my bottleneck is scraping the images since I don't have terrabtytes of bandwidth and imgur would likely rate limit me. But given the
by edude03 10y ago
I thought of doing this but my bottleneck is scraping the images since I don't have terrabtytes of bandwidth and imgur would likely rate limit me. But given the data set my plan is/was to extract tags by mapping images to the sub Reddit they're posted in then categorize based on that
- i336_ 10y agoContabo are 100Mbit but unmetered, ridiculously cheap. Reddit's main issue is that you can only do one call (via OAuth) per second. I recommend using the new(ish) "thing" call where you can get up to 100 things at once, or alternatively rockets.cc, which occasionally falls apart but is pretty good
- minimaxir 10y agoThere's no reason to scrape Reddit for the image URLs when you can download all NSFW image links on Reddit via BigQuery.