3 ms·
You can read the rest of my MJ12Bot saga: http://boston.conman.org/2019/07/09-12 http://boston.conman.org/2019/07/09-12 My take: they are grossly incompetent
by spc476 7y ago
You can read the rest of my MJ12Bot saga: http://boston.conman.org/2019/07/09-12 http://boston.conman.org/2019/07/09-12 My take: they are grossly incompetent at programming.
- syxun 7y agoThe best robots.txt for Majestic: iptables -A INPUT -s 207.244.157.10/32 -j DROP
- spc476 7y agoIf only it was that easy. Last month MJ12Bot hit my site from 136 distinct IP addresses. If we drop the last octet, it's 120 unique class-C addresses, and if we drop the last two octets, then 43 unique class-B addresses (and why not---31 distinct class-A addresses). It's a distributed bot. Very hard to block, so I think I came out ahead by them no longer spidering my site. Edit: Added count of class-A blocks.
- Double_a_92 7y agoTurns out that if one wants something from people, one shouldn't start the first interaction by calling them "grossly incompetent"...