4 ms·
right now we're getting .au domains from Open Crawl. It's tricky because the zone file is not available like you mentioned. If anyone has any ideas on how to
by mxpxrocks10 9y ago
right now we're getting .au domains from Open Crawl. It's tricky because the zone file is not available like you mentioned. If anyone has any ideas on how to get more .au domains, we'll gladly implement.
- jlgaddis 9y agoCT logs come to mind, if you aren't using them already. DNS queries, if you have access to recursive resolvers.
- mxpxrocks10 9y agowe're using CT logs for getting hostnames (you can see it at the top of the page of any domain). We have a great recursive resolver source but haven't been able to integrate it into the data pipeline yet. thanks for the note and ideas!
- hughesey 9y agoThere's some reverse engineered zone files for a lot of cctlds at http://viewdns.info/data/ http://viewdns.info/data/.