3 ms·
We are currently crawling all domains (in the tlds we cover) once a month, and so it should be resolved automatically at an average of two weeks. The feedback
by epi0Bauqu 18y ago
We are currently crawling all domains (in the tlds we cover) once a month, and so it should be resolved automatically at an average of two weeks. The feedback on the site is to help us detect false positives and get people unblocked quicker, where appropriate.
- ejs 18y agoOk great, I was hoping so but did not see it anywhere spelled out. Good to hear!
- epi0Bauqu 18y agoIt's actually on the report a false positive page (http://www.ivegotafang.com/fp.html http://www.ivegotafang.com/fp.html), but I agree it should be made more clear on the front page as well. Thanks for the feedback!
- katz 18y agoNaive questions: How does your data center look like? What is your web crawler written in? I assume that your startup is fairly small - how difficult is/was it to get enough resources (funding) to scale up? (esp. datacenter wise) I see that you do not just cover English and other (co.?? org.??) domains are covered fairly well.
- epi0Bauqu 18y agoThe crawler is written in Perl, and has been optimized such that the amount of resources we need to produce our list is actually not that much.