2 ms·
Show HN: I built a tool to scrape and summarize Reddit long discussions
I built this because I spend too much time digging through thousands of comments on Reddit to find the actual conclusion or consensus.
First I used manual approach:
- add .json to URL to get metadata (fetches ALL comments)
- pass json to AI to get the summary
It works, you can use it, but metadata are "dirty", containing lots of unnecessary fields that bloats up the context for AI. And it's not that fast, I wanted single click solution.
It wasn't hard to automate it in chrome extension.
Supported pages are Threads, Subreddits, Search results.
It’s built with Plasmo and React. The backend is a serverless proxy (Vercel) to handle rate limiting, key security and calls to OpenRouter API.
Demo: https://youtu.be/9MGwQElbBXE https://youtu.be/9MGwQElbBXE
Chrome extension: https://chromewebstore.google.com/detail/jccgjnbkibbigimcfohndjbakjhelfoo?utm_source=item-share-cb https://chromewebstore.google.com/detail/jccgjnbkibbigimcfoh...
Feedback is welcome!
- SilverElfin 11mo agoHow do you get around the rate limits sites usually have to stop scraping?
- MartinMachava 11mo agoIt's just one request per analysis. Reddit allows much more. I thought about that when I thought implementing "deep mode". Right now when user analyzes: - Thread, it fetches all metadata, 1 req - Subreddit, fetches 100 posts, 1 req - Search result, 100 posts, 1 req Subreddit posts and Search result posts are just posts. Extension doesn't open the posts to get all comments of each. But I thought of it. I don't remember exact numbers of reddit's rate limits but it should be possible to implement in the future. I'm thinking of adding "overheat bar" (like in games) to indicate to the user how much requests is remaining per minute and other safety checks. Fetching metadata is done on user's behave, not reddit API. It's not scraping. Metadata are accessible to anyone by just adding .json to URL
- MartinMachava 11mo agoI just launched it on Product Hunt. If you like what I built (and only if you like it), support me with a vote there. I'd be happy to get 5 votes. https://www.producthunt.com/products/reddit-summarizer?utm_source=other&utm_medium=social https://www.producthunt.com/products/reddit-summarizer?utm_s...