4 ms·Isn’t this easy for LLMs to avoid by passing an instruction to ignore any hidden links ?by snehesht 11mo agoIsn’t this easy for LLMs to avoid by passing an instruction to ignore any hidden links ?krackers 11mo agoCompanies mass crawling don't use LLMs for crawling itself, that would be too expensive.snehesht 11mo agoMake sense, but doesn't necessarily have to be an llm, just a regular dom parser will be able to tell whether an element is visible or hidden.