3 ms·
> When I use services like Reveddit I often notice [removed too quickly] where I'm guessing moderators get to it before the scraper can archive it. What type of
by rhaksw 4y ago
> When I use services like Reveddit I often notice [removed too quickly] where I'm guessing moderators get to it before the scraper can archive it.
What type of timeframe do you guys typically work with before it gets removed? Any ideas on how to improve the capture times?
Current capture time is noted in the status on Reveddit.com/info [1]. That blue box will show up in threads when the capture time is delayed more than the standard, which I think is typically under a minute.
The content for [removed] comments in threads comes from another third party service called Pushshift. It isn't able to get many auto-removed comments (such as those removed by Automoderator), and that service sometimes goes down, may fail to ingest something, etc.
In an attempt to fill some of these gaps, Reveddit can restore some unarchived comments by searching the user pages of other commenters in the thread. And if you visit Reveddit via a direct link to a removed comment, such as from the button that is provided under such comments by the Reveddit Linker extension [2], then it will try to auto-load those comments this way. Otherwise you can click Restore to do it manually.
Almost all of the features on the site should be documented in either a bubble or the FAQ. If you see something that you think needs a help bubble, or a better description, let me know.
> Also are you looking for dev help with your project?
Sure! The code is on Github. Feel free to brainstorm via issues, email, or submit PRs. My main focus now is raising awareness so it would be great to have backfill on keeping the code functional.
[1] https://www.reveddit.com/info https://www.reveddit.com/info
[2] https://www.reveddit.com/add-ons/linker https://www.reveddit.com/add-ons/linker
- dmix 4y ago> Reveddit can restore some unarchived comments by searching the user pages of other commenters in the thread. Clever
- rhaksw 4y agoThere are a lot of fun things you find when you start working on this. Virtually nobody else is doing it because they all think it's wrong without realizing it's the right thing to do. For example, you could reverse engineer automod configurations by scraping user pages with a headless browser, storing all of the auto-removed comments, and then looking for the most common n-grams in a given subreddit. Alternatively you could make the code work "server-side" so some sort of json is returnable, though that would take more work. And I haven't even mentioned the history pages, which may indicate where users and moderators have disagreed the most: https://www.reveddit.com/v/worldnews/history/ https://www.reveddit.com/v/worldnews/history/ Plus, this is only one social media site. Every site does this, and there's probably a way to reveal the shenanigans going on in a lot more places.