4 ms·
searching for .gov reveals 0 matches... doubt
by cvadict 3mo ago
searching for .gov reveals 0 matches... doubt
- sandeepkd 3mo agoMy guess is that they ran selective search on the domains which get registered with any registrar, thats the trigger to start the search. .gov domains are not managed by your typical registrar which is selling the domain registration information to all these downstream partners/scavengers (for lack of better word)
- jadamson 3mo agoThe OP says it's using CT logs, not new domain registrations. The approach you have in mind would not include subdomains and would be less likely to coincide with a new server being configured.
- sandeepkd 3mo agoYes CT is explicitly stated source which is why I qualified it with "Guess" for domain registration. There were couple reasons for that - 1. Quite a few websites in the search results where just on HTTP 2. The .gov sites do use public certificate authorities like digicert, verisign, amazon & letencrypt so they would have been captured unless they are removed explicitly And yes the domain registration would not include subdomains
- marysol5 3mo ago.gov.uk plenty of them use standard CA's manchester.gov.uk uses LE
- PatchRequest 3mo agoI manually blacklisted .gov from being crawled on my side. felt like it wasn't worth the potential trouble
- marysol5 3mo agoMeanwhile you crawl anybody else? .gov isn't magically any more able to take you to court than a private org