10 ms·
Find every domain someone owns automatically
- moonka 9y agoThat is very handy.
- stanislavb 9y agoAlthough it didn't work as I expected.
- mxpxrocks10 9y agoHey, we're still kind of new and kicking the tires. Any UX feedback or data feedback is appreciated!
- regecks 9y agoIt's pretty much missing or empty data for the entire .au TLD (e.g. even google.com.au is not present). Living here, I know that their whois server rate limits heavily and the ccTLD zonefile is not available, so I'm guessing those are probably contributing reasons. Do you guys do any crawling at all? I think this is a fantastic concept though. Knowing that whois database and zonefile access is often protected for commercial motivations, it really irks me. Open it all up.
- mxpxrocks10 9y agoright now we're getting .au domains from Open Crawl. It's tricky because the zone file is not available like you mentioned. If anyone has any ideas on how to get more .au domains, we'll gladly implement.
- jlgaddis 9y agoCT logs come to mind, if you aren't using them already. DNS queries, if you have access to recursive resolvers.
- mxpxrocks10 9y agowe're using CT logs for getting hostnames (you can see it at the top of the page of any domain). We have a great recursive resolver source but haven't been able to integrate it into the data pipeline yet. thanks for the note and ideas!
- hughesey 9y agoThere's some reverse engineered zone files for a lot of cctlds at http://viewdns.info/data/ http://viewdns.info/data/.
- mxpxrocks10 9y agowoot- glad you like it so far. Any actionable feedback greatly appreciated.
- moonka 9y agoIt worked exactly as I hoped. I couldn't figure out why some domains I were expecting to show up weren't, but searching by email reminded me that I need up update some of my whois info due to a name change.
- mxpxrocks10 9y agoawesome! any feedback or ideas are appreciated, thanks for trying it out.
- laurencei 9y agoJust tried it with some of my own details. Found a domain I forgot I owned on an old reseller I havent used in a while - lol. On a more serious note - I'm very curious how you get such a long history of domains. i.e. I can see every DNS change and ownership for any domain - I didnt realise that was always available?
- mxpxrocks10 9y agoHi! We acquired 4 companies that have been doing lots of cool data work. We also license and collect our own data to mix in. There's different granularity depending on the data (WHOIS history, Name Server history, DNS record history, technologies used) etc. We're constantly improving it. The thesis we have is that if you get hacked, it often times is through an old server or satellite domain. We're building tools to help you find the extended surface area where you can be hacked or have downtime. The example of you finding an old domain is a prefect use case.
- jsjohnst 9y agoJust out of curiosity, was one of those companies originally named DeletedDomains.com? They had the full root zone (of the ones now managed by Verisign GRS anyway) since approximately 2001 (my involvement with them was 2003).
- mxpxrocks10 9y agohey - no. wwws.io, dnstrails.com were the main ones for this data
- jgh 9y agoi guess this doesn't work for private registration. E.g. Domains by Proxy's phone number has ~11 million domains.
- mxpxrocks10 9y agoWe have a really fun blog post coming out for this next week or so. One of our team found a pretty big bug in certain private registrations. Stay tuned.
- arca_vorago 9y agoYou say fun, but in the era of swatting and doxxing, private registrations are often the first line of defense for those who don't have a USPS or ups mail address if they don't have a physical office/address. I would be very careful about how you responsibly disclose any such bugs.
- mxpxrocks10 9y agoGood point. Poor word choice on my end. Thanks for the note. The stuff we found is not quite that abrasive but very interesting. How would you recommend disclosing?
- mynewtb 9y agoLet the providers fix it. Contact the people potentially affected if you can.
- ploggingdev 9y agoCan you give a gist of your findings? I'll take a guess : customers are given unique whois protected email addresses allowing you to find all the domains owned by the person. Eg- If I own abc.com and xyz.com, both have the same public email address. The problem with such bugs is that there's no way to undo the damage since historical whois records are archived.
- 9y ago
- wiradikusuma 9y agoIt doesn't work for privacy-protected whois (obviously), but it also doesn't work for .id domains (try http://every.id http://every.id, http://awesome.id http://awesome.id or http://player.id http://player.id).
- mxpxrocks10 9y agoWe'll add .id to the wish list. Tricky because they don't make the zone file avail, but we'll figure something out. Thanks for the note.
- kuschku 9y agoSo, for .de it just returns either NULL, undefined, or empty for everything, for .eu it errors out entirely, and for the domains of me it does find, it has wrong data. kuschku.de has, for the past 2 years, always pointed at 51.15.1.223 or 163.172.217.134, never at 204.236.227.242. Funnily, for other domains pointed at the same IPs, it has correct data – e.g. quasseldroid.info correctly shows the IP history. The datasets used here seem of questionable quality, souring the taste of this awesome feature.
- mxpxrocks10 9y agohey- would love it if you can drop me an email (in profile) so we can debug/improve. .DE is particularly hard because they lock down the zone file. GTLDs like .INFO are easier to get because the zone files are open. We have 9 years daily granularity for the gTLDs. Would love to clean up the U/X so it's clear what we have data for and what we don't to be completely transparent. Thanks for the note.
- kuschku 9y agoYour profile doesn’t have any email address in it, but you can just check https://dnstrails.com/#/domain/domain/kuschku.de https://dnstrails.com/#/domain/domain/kuschku.de vs. https://dnstrails.com/#/domain/domain/quasseldroid.info https://dnstrails.com/#/domain/domain/quasseldroid.info For example, WHOIS info https://i.imgur.com/WNpyvcl.png https://i.imgur.com/WNpyvcl.png should maybe show something like "none available", or "no WHOIS info is available for .de", or "go to denic.de to see WHOIS info" (DENIC offers the WHOIS info, if you enter the captcha). On the other hand, http://whois.domaintools.com/kuschku.de http://whois.domaintools.com/kuschku.de (a competitor) correctly shows the WHOIS. Second, with the domain – I have no idea how the wrong value ended up on there.
- computer22 9y agoHi kuschku, thanks for providing us with the samples with the outdated/missing data. We will definitely look into the case and continue to enhance the data we collect. The UI enhancements you mention are excellent - we will implement this shortly (not only for .de but for any case where we can not output any values). Feel free to contact us at the e-mail address given at the bottom on every dnstrails page!
- bananamansion 9y agosearched google.com. turns out they own the url.. android.porn
- computer22 9y agoNice find bananamansion. Actually you will see this with many big cooperations which also register domains such as <companyname>.sucks, <companyname>.adult etc. ;)
- IncRnd 9y agoThey have that domain so that nobody else uses it.
- Jaruzel 9y agoI have a few of those. :)
- brad1043 9y agoI know that GoDaddy blocks WHOIS requests like crazy. How are you able to 'bypass' that restriction at such scale?
- mxpxrocks10 9y agoHi. There's some good info online. https://www.icann.org/resources/pages/approved-with-specs-2013-09-17-en https://www.icann.org/resources/pages/approved-with-specs-20... Check out section 3.3.6
- brad1043 9y agoAlso, is your WHOIS data for each domain "live", meaning it's being queried in real time?
- mxpxrocks10 9y ago1. for the current record displayed on the domain results page, it's live but then has a short cache after the first time it's pulled on the public site. 2. For the whois registrant search, it's around 90 days old right now, but we're working on techniques to make it more current.
- Hasz 9y agoCouldn’t get it to work with .rs domains. That being said, cool tool!
- mxpxrocks10 9y ago<sigh> these country code domains. LOL. We bought and built up http://www.domainlists.io http://www.domainlists.io so we can add some more stuff like this in. I'll make sure we add .RS to the action items. Thanks for the feedback and enjoy!
- toomuchtodo 9y agoHow do you plan on handling the EU’s “right to be forgotten” (it’s pretty straightforward to make the argument you’re a search engine) and other components of the GDPR?
- mxpxrocks10 9y agoHaven't really explored this. Any feedback or ideas is appreciated!
- ratherbefuddled 9y agoYou should read up on GDPR urgently. It applies to anyone anywhere in the world processing data of EU citizens and has some massive penalties for things like not asking for specific informed consent and not offering an opt out. Using the excuse that you're just aggregating public data does not cut it.
- togus 9y ago"The right to erasure" is not an absolute right for anyone to get all their data deleted. If the data owner (read: the registrars) still have a legal right to collect and maintain the data public and it has not been revoked one could argue that they (security trails) don't have to remove the data. It's my understanding that the registrars are the ones with the burden here. They need to inform everyone of the data erasure and/or data updates on private information. Fun times when you have public information for anyone to gather on the internet. It could be that there are exemptions for these kind of services, I do not know, but would the exemption not also include the services that aggregate/collect historic information as well? Disclaimer; I am not a lawyer. I am not well versed in GDPR. Anyone finding this interesting should go read up on GDPR.
- heinrich5991 9y ago>one could argue that they (security trails) don't have to remove the data. It doesn't work that way. The "right to be forgotten" can be used to remove search results from Google, even if the original content stays up.
- graysonk 9y agoHmm is there a technical write up of how you are pulling this data? I tried one of our companies for fun and it’s only pulling 1.4million records in one place and then 65,000 in another. Doesn’t seem to have all our nameservers or relays either.
- mxpxrocks10 9y agoWould love to hear more so we can debug. chris at securitytrails.com if you have a few mins so we can chase things down. We'll do a technical blog post soon - good idea.
- jsjohnst 9y agoBased on the supported TLDs, I’m guessing they are pulling down the root zone tables from Verisign GRS. Verisign licenses the Whois data in bulk out to companies like theirs.
- vxNsr 9y agoI'm finding that sometimes it'll turn up the data correctly and other times it won't match what I can find by manually typing in addresses into domains.google As in this service will claim all data is private when google is able to return the actual registrant email address and/or name. As well as valid phone numbers which don't match what dnstrails is outputting.
- mxpxrocks10 9y agothanks for the note and testing. Would love to dig into any specific use cases so we can see whats going on. chris at securitytrails.com
- sebazzz 9y agoTry damsteen.nl for instance, my blog, it shows no whois at all.
- mxpxrocks10 9y agogot it - thanks!
- megous 9y agoMust say, I'm not a fan of yours or similar services, or whois databases for that matter. Privacy should be easier on the internet for people owning domain names.
- rosstex 9y agoSecurity through obscurity.
- megous 9y agoYou probably misread the word. It's not security, but privacy. And privacy is made by obscuring, usually.
- mxpxrocks10 9y agoHi! The only way on the current Internet to remain anonymous is to obscure the domain registrant from right when the domain is initially registered. It's also important to use a service that pools a lot of domains together or have diversity in the hosting providers so they can't be easily correlated. We try and help people understand and make the problem visible. You bring up a good point that it should be easier.
- megous 9y agoYeah, it's hard. If one has multiple domains from multiple tld's pointing to the same server, it's really easy to find out other domains and tlds with more detailed and easier to access records that will reveal the owner's identity. Privately registered eu domains are quite nice. I couldn't even find two of such that I own in your database and whois only reveals email (which I made sure from the beginning to be of the same domain that I just registered, so that's useless for anything). People can probably pay for more info though.
- starsinspace 9y agoA good option for people living in the EU is to register a .eu domain. For private individuals (like private non-commercial websites), the whois data is by default restricted to show an e-mail address only. (see section 2.4. in https://eurid.eu/en/other-infomation/whois-policy/ https://eurid.eu/en/other-infomation/whois-policy/)
- ohashi 9y agoI think http://whoisology.com http://whoisology.com has been doing this for ages.
- computer22 9y agoHi ohashi, we do not only track the current and historical whois records, but also current and historical DNS records, even for subdomain, which technologies the website uses and even more data which we are currently working on. Maybe try our WHOIS aggregation feature and let us know what you think!
- meowface 9y agoSo has Domaintools (http://domaintools.com http://domaintools.com) but both are expensive (Domaintools in particular is very expensive) whereas DNSTrails appears to be free unless you want API access.
- hughesey 9y agoSo has http://viewdns.info http://viewdns.info. Free too.
- qubex 9y agoGreat resource, thanks.
- qubex 9y agoCan't even try it on one of my own domains (.IT) without paying up first...
- CodeWriter23 9y agoLooks like I own around 289,000+ domains that I’m not aware of.
- mxpxrocks10 9y agoawesome. hopefully your creditcard isn't on file for the renewals :-) . we'd love to hear about any bugs- chris at securitytrails.com
- shadowashe 9y agoBinaryEdge does something like this but also for IP addresses and then security rates them https://blog.binaryedge.io/2017/11/23/organization-mapping/ https://blog.binaryedge.io/2017/11/23/organization-mapping/
- tzs 9y agoThe results page should always show what your search term was. Currently it only shows this when the search finds something. If nothing is found, all it does is tell you that nothing was found. That leaves you with no way to check to make sure you entered the right term, other than typing it again. If you are typing it right but auto-correct is kicking in when you hit enter, even trying again might fail because your site responds fast enough that you might not have time to notice the change before the results page comes up. Also, it might be helpful when someone searches for a domain at a TLD that you do not support to say on the results page that the TLD is not supported.
- scrollaway 9y ago+1 on search term! And if you enter a domain that doesn't exist, it throws you back to the homepage with no way to fix it :( Another feature request: When entering an IP, there's very little information available on it (only the hosting provider). Would be nice to get latlong, country etc (a la https://www.iplocation.net/ https://www.iplocation.net/). Because seriously, that is an insanely cool database. I can even find all the sites that share my domain's Cloudflare IPs.
- mxpxrocks10 9y agoawesome. thanks for the feedback! great idea, we'll implement this week. Would love to stay in touch.
- mxpxrocks10 9y agoAwesome ideas. Will implement. If you'd like updates in the future drop me a line at chris at securitytrails.com - would love to have you keep testing.
- richardkeller 9y agoThis is great. Is there are a list of supported TLDs? For example, I'm not having much luck with .co.za (South African) domains.
- qubex 9y ago.IT (Italy) neither.
- rootsudo 9y agoScary, yet useful as fuck.
- qubex 9y agoThis is brilliant. But a question: why is one website I know of (and that resolves, and that has valid DNS entries) comes up blank?
- mxpxrocks10 9y agohey, thanks for checking. Shoot over what you're looking at to hello at dnstrails.com and we'll investigate.
- blixt 9y agoThis can't be "The World's Largest Repository" when half my domains are "not a valid domain" according to the tool.
- noja 9y agoIt's like Uber all over again. This is expressly forbidden with most registries, especially European ones.
- greggman 9y agoI didn't look into what this is doing but it's not finding 1/2 of my domains (they are not private). I'm guessing it's not "Find every domain someone owns" it's "Find every domain that meets X criteria" which may more may not be every domain someone owns.
- scaryclam 9y agoYeah, I was fairly unimpressed when it didn't even manage to lookup the first domain I entered (it's not new and it's not private). I guess it might be useful for some domains, but there's no way I'd rely on it.
- mxpxrocks10 9y agoSame about this one. Any further info so we can enhance the user experience would be awesome. We have a tough time still with ccTLDs because registries locking down the zone files. Any more info or suggestions appreciated.
- mxpxrocks10 9y agoHi! Would love to know more about the domains it's not finding. Are they Country Code domains or gTLDs? Any other info so we can debug would be awesome. Thanks for trying the service!
- dws 9y agoI have the opposite problem. It's finding 1300+ domains that I don't own. The perils of name matching against a common name.
- techbubble 9y agoIs it possible to include WHOIS data from way back when InterNIC was the only registrar? I own a domain I registered way back when you would send an email to InterNIC and registration was free. The WHOIS data returned for the domain only starts in 2008 and skips about 12-14 years.
- mobilemidget 9y ago\o/ I own 5 million domains... wait a minute do I pay for all of those?!
- brianwawok 9y agoYah same for me on google domains
- mxpxrocks10 9y agohey, can you give a little more info? Are you sure it's not picking up on some other part of the registrant?
- bpicolo 9y agoWhoisguard variants.
- empressplay 9y agoGoDaddy is now masking _all_ DNS info whether you pay for their "privacy" service or not, so thats making a substantial black hole in this data
- epalmer 9y agoIt says I own 86 domains but is using my given names and not my email address. My name is not unique. Hardly a valuable service. I own less than 10 for the record.
- mxpxrocks10 9y agosuggestions on how to make it more effective for your use case?
- wimagguc 9y agoWhat’s interesting is that it correctly finds Namecheap whois-guarded domains too. If I search for the guarded domain directly it shows the correct record as the owner’s address/etc being WhoisGuard, but then if I search for a non-guarded domain and click through from the identified name, it does list the guarded domains as well (!)
- mxpxrocks10 9y agocan you give an example?
- nerdponx 9y agoThe OP said something about zone files earlier, so it's possible that they're getting their data from more than just Whois lookups.
- deleted 9y ago[deleted]
- puppetmaster40 9y agoVery nice! Also... scary.
- nerdponx 9y agoIs this just a reverse indexed WHOIS database? If so, it's no surprise that my domains don't appear in yhe results: I signed up for Whois anonymization through my DNS provider. I was recently considering unsubscribing, so thank you to the creators of this for reminding me that my privacy is under attack at all times and I should do whatever I can to protect it.
- Jach 9y agoThat's what I think too, and I definitely don't own millions of domains. Be careful about trusting privacy protection plans. A spam email came to me recently showing the unmasked info and I still have no idea how it acquired its data... https://www.thejach.com/view/2017/10/google_whois_protection_doesnt_protect https://www.thejach.com/view/2017/10/google_whois_protection...
- markdown 9y agoIt said I owned 45 domains, and listed 10 that I actually own along with 35 I've never owned, heard of, or ever been associated with. I have a very common english name. Annoyingly, some of the domains it inaccurately says I own are NSFW. They need to put a big disclaimer on the results page pointing out that the results aren't necessarily accurate.
- bartvk 9y agoI'm pretty sure this goes against European (or at least Dutch) privacy laws because it's not just company domains being searched. There isn't any privacy-overriding reason to keep a database with this kind of identifying information. Since these laws are currently barely enforced, nothing will happen of course. More info in Dutch: https://blog.iusmentis.com/2017/11/08/internationale-domeinbeheerder-staat-afschermen-domeininfo-toe/ https://blog.iusmentis.com/2017/11/08/internationale-domeinb...