4 ms·
I scrape about 60 blogs and news sites that deal with a niche topic and examine all the hyperlinks. If more than one of them links to the same page, I assume t
by jawns 12y ago
I scrape about 60 blogs and news sites that deal with a niche topic and examine all the hyperlinks. If more than one of them links to the same page, I assume that it's a page that's generating some buzz, so I send it out in an email. It's proved to be a generally reliable assumption.
- Torgo 12y agoWould love it if you posted a blog article sometime about the technical details.
- jawns 12y agoThere's really not much to it. Scrape each site/feed every X minutes, find all the hyperlinked URLs on the page, add them to a database table ... and if they're already in the table, send out an email with links to the "buzzed about" URL, as well as all of the sites/feeds that mention it. I keep the links in the table for about a month.
- stephancoral 12y agoHave you thought about the case in which a blogger might be using a URL shortener? You might be missing some potentially 'buzzy' links if you are just looking at the direct URL string