4 ms·
Ask HN: Where can I find Internet browsing data?
Somebody posted a large dataset from Indiana University a while back, but it's several years old and only available to researchers.
http://cnets.indiana.edu/groups/nan/webtraffic/click-dataset
Does anyone know of another source of browsing data? Is it possible to get this data in real-time?
- Irishsteve 13y agoTo get it in real time you most likely have to pay someone who is already collecting said data (Don't know who) or write software yourself and pay people to install it. Is there any particular type of data you are looking for? Or is it simply anon user x went to page y on domain z
- gregw134 13y ago>you most likely have to pay someone who is already collecting said data (Don't know who) or write software yourself and pay people to install it. Yeah that's what I was afraid of :( I'm looking for basic, anonymized browsing history for users over a period of time. The idea I'm trying to test is whether you can identify the users who are "trendsetters"--people who visit sites or posts before everyone else--and whether you can sniff out links that will be popular based on their behavior. I've found a few more links, but most of them require you to be a researcher to access the data: https://www.predict.org https://www.predict.org http://www.caida.org http://www.caida.org http://crawdad.cs.dartmouth.edu/ http://crawdad.cs.dartmouth.edu/
- ig1 13y agoWhy not do it based on tweets ?
- Irishsteve 13y agoSpotting trend setters on the web is going to be a little tough because of how large and segregated it is. Some datasets people use is Wiki based around comments / edits or who created a page first which became popular in the end. You could also use Facebook likes (facebook.com/me/likes) where each hlike is time stamped.