3 ms·
Yes, that is the work of crawling and parsing. On some sites, they are human edited as well.
by epi0Bauqu 19y ago
Yes, that is the work of crawling and parsing. On some sites, they are human edited as well.
- amrithk 19y agoAny resources on how to do this?
- epi0Bauqu 19y agoWell, I am doing this myself at the moment. I use LWP in Perl to crawl. As for generating the description, it depends on what you want to do. You can use combinations of the title and meta tags, or if it is just major sites you want, try to grab a (at least partially) human edited form like from Alexa. What is the application?