3 ms·

Show HN: An open source web crawler for the Mwmbl non-profit search engine

• Komodai 4y ago
  Thought it would be an actual web crawler from the title, got really disappointed.
  • jka 4y ago
    Good news: it looks like the server-side crawler code that the browser extension sends content to is open source too: https://github.com/mwmbl/crawler-server https://github.com/mwmbl/crawler-server
    • Komodai 4y ago
      Ah ok.
  • daoudc 4y ago
    It is an actual web crawler. I should have put a link to the server side code.