3 ms·
Thank you very much :) Scalability: This is more of a cost/resource issue than a technical one. Right now, ForumScout checks every hour for new posts. This isn
by joshwaller7 2y ago
Thank you very much :)
Scalability: This is more of a cost/resource issue than a technical one. Right now, ForumScout checks every hour for new posts. This isn't ideal, but it allows me to offer the product for much cheaper than if we were to check, say, every 5-10 minutes. Scalability is much more manageable with an hourly check, but the goal eventually is to get as close to real time as possible. I plan on offering a higher pricing tier for real-time checks to account for the extra resources.
Filtering: By default, the searches are performed via exact match. From testing, there are no false positives; there's some code that double-checks that each post contains an exact match of the keyword(s). Users can manually turn off exact match if they want broader searches. Sentiment searching is a VERY good idea. I will add it to my list!
Integrations: This is a great point, and something I had not yet considered. Right now, the feed can be exported via CSV, but direct integrations would be much better for UX.
Technical: Honestly, it wasn't too difficult - every hour, the automation will make a call to all the different APIs and search using the parameters provided by the user at setup. Then we format and combine all the results, remove duplicates, perform some more filtering (depending on what settings the user has chosen), and add the results to the user's feed. If emails are turned on, we send an email with the new results. As mentioned, making this real-time will be more difficult, but it would really just mean doing this every few minutes, requiring more resources. I did build a custom scraping function for Reddit comments, because the native Reddit API doesn't allow direct searching of comments like you can on their website/app, but everything else is third-party.
- msnkarthik 2y agoAwesome