3 ms·
Sorry. I think I totally botched this post. I'm looking for a way to automatically look up any sites linking to a given URL. I'm building a memetracker and w
by brandnewlow 18y ago
Sorry. I think I totally botched this post.
I'm looking for a way to automatically look up any sites linking to a given URL.
I'm building a memetracker and while I've got the related stories and clumping of them in hand, tracking down the discussion links in the way that Techmeme does it is proving much harder to do.
I was wondering if there was a way to do it other than Cosmos. And if not, what would be involved in creating something that does that? Building a web spider?
- mikeyur 18y agoI think Techmeme just crawls the sites in their 'network' for cross-linking. For example Techmeme has TechCrunch/Ars/Engadget/Scobleizer/etc in their network and if they link within each other then that pushes the story up higher (again, not 100% sure, just making an assumption based on what I've seen). I don't think there is a real-time way to see how many sites are linking to another unless they're being crawled quite often by google/yahoo/technorati OR you have a script installed on their site to see the inbound links and gauge traffic.