4 ms·

Thanks, mate! It crawls only projects mentioned on Reddit, Hacker News and Dev.to.

• ronyfadel 5y ago
  Crawling noob here: what's a good strategy to crawl these websites? Reddit is huge, how does one efficiently crawl it? Search API for Github links?
  • stanislavb 5y ago
    I've built a custom engine to achieve that (Elixir/Phoenix). It uses a simplified version of the Aho–Corasick algorithm https://en.wikipedia.org/wiki/Aho%E2%80%93Corasick_algorithm https://en.wikipedia.org/wiki/Aho%E2%80%93Corasick_algorithm to match all mentions of subscribed keywords/URLs. I use the same engine for SaaSHub, too.
• jspann 5y ago
  I see. Nice work!
  • stanislavb 5y ago
    Cheers! Work appreciation here on HN is highly appreciated. (as that could be a rare beast)