3 ms·

Pretty sure twiceler is Cuil....

• jacquesm 17y ago
  Ah! thanks. That clinches it then. One more agent for the ban list. Apropos, I think that plenty of bad guys are masquerading as twiceler as well. Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html http://www.cuil.com/twiceler/robot.html When 'CUIL' first came out their crawler was all over my servers, sometimes to the point of making them inaccessible for users, I remember having a devil of a time getting rid of them because they came from lots of different IP blocks. Apparently their bot is called twiceler, maybe they should name it 'CuilBot' or something to that effect.
  • whughes 17y ago
    Judging by your response, maybe they shouldn't name it Cuilbot. I wonder if their overzealous bots and subsequent blocking are reasons why they launched with such terrible results. I heard several reasons for that catastrophic failure: * Overstressed subject-specific servers failed, leading to porn on hamburger searches and that kind of thing * Crashing, random result variations due to overload * Excessive media hype exacerbated the load issues Another one to add to the list?
  • eli 17y ago
    mod_security makes easy work of these sorts of rules