4 ms·
This is awesome. Thanks. Where do you query the data from? Do you scrape it from somewhere and store it in your own database?
by jazoom 9y ago
This is awesome. Thanks. Where do you query the data from? Do you scrape it from somewhere and store it in your own database?
- mcrowe 9y agoThanks! You can request access to zone files from the domain authorities. I take the raw zone file and load it into a series of Bloom filters backed by Redis. There are a lot of domains to check against, and a probabilistic data structure is the only way to store it in memory, which you need for performance. This has the small caveat that the tool will accidentally think a domain is not available about 0.1% of the time. However, it shouldn't ever report domains as available when they are not. So that works quite well for a tool like this.
- jazoom 9y agoVery impressive. I will be using this in future.