3 ms·
Companies mass crawling don't use LLMs for crawling itself, that would be too expensive.
by krackers 11mo ago
Companies mass crawling don't use LLMs for crawling itself, that would be too expensive.
- snehesht 11mo agoMake sense, but doesn't necessarily have to be an llm, just a regular dom parser will be able to tell whether an element is visible or hidden.