ghafouri

3 ms·

1. 1.▲
   by ghafouri 2y ago

   With reverse DNS, I only get a few results, not hundreds of thousands of domains. So, I took this approach.
2. 2.▲
   by ghafouri 2y ago

   I used to work on a Shopify SaaS project, but it was canceled. I'm thinking of sharing some of my work from that project.
3. 3.▲
   by ghafouri 2y ago

   I achieved 1000 DNS resolves per second with DOH, and my attempts on DNS servers are fewer.
4. 4.▲
   Web Crawl Analysis: Unveiling 465K Shopify Domains from 3TB of Data(alistechtales.substack.com)

   7 points by ghafouri 2y ago|10 comments
5. 5.▲
   by ghafouri 2y ago

   Dived into Common Crawl’s 45 billion URLs, processing 3TB of data using Python's aiohttp and multiprocessing, Kaggle, and MinIO. Resolved domains with Google’s and Cloudflare’s DNS over HTTPS, uncovering over 465,000 Shopify domains. D