3 ms·
Have you ever heard of https://commoncrawl.org/ https://commoncrawl.org/? Maybe it helps you to bootstrap. It's cool to see some new things happening in the se
by ericst 8y ago
Have you ever heard of https://commoncrawl.org/ https://commoncrawl.org/? Maybe it helps you to bootstrap.
It's cool to see some new things happening in the search engine world.
- brentadamson 8y agoI've considered that but didn't really know anything about the organization and how long they'll be around. We're gonna give Colly a try: https://github.com/gocolly/colly https://github.com/gocolly/colly.