2 ms·
I will try this out over the next day or so and let you know my thoughts. Can I ask how you did the crawling to get this started?
by Topology1 2mo ago
I will try this out over the next day or so and let you know my thoughts. Can I ask how you did the crawling to get this started?
- wesammikhail 2mo agoAppreciate it. Really looking forward to your feedback. I have my own infra setup and started with 25k seed domains including major known startups, YC companies and things like that. I collected some of the data via scraping and then merged it with 3rd party enrichment sources. Once the crawling per domain is finished, it is put on a 30 day schedule to repeat the crawl just to keep things fresh. If you are on desktop and click on a result entry, you'll see expanded data for each domain. In reality that's just a small portion of the total data I collect. Anyway, really looking forward to hearing your thoughts!