4 ms·

Ask HN: Review My Doohicky

19 points by ajkirwin 18y ago

I got bored this weekend and decided I wanted to data mine twitter and over the course of a day and a half, I made this: http://malicelabs.com/linkalyzer/ and I would like suggestions from people as to how to improve it and what they think of it. :) Edit: Intermittently broke things and blew away some data intentionally. Add(ing|ed) in tinyurl/is.gd/etc through-finding of real URLs and page title grabbing.

gsmaverick 18y ago
Neat analyzer! I think there is so much data that can be gathered from Twitter that we have no idea exists.
riklomas 18y ago
Maybe try and reverse engineer the TinyURLs so that users can see where they link to?
- iamdave 18y ago
  This. Or maybe find a way to grab the page <title> from the target URL and create a separate column with the appropriate information. Otherwise, nifty little app.
nirmal 18y ago
Would be good to expand the tinyurls or at least provide the domain they are pointing too. This may even help you consolidate tinyurls if multiple services are used for the same original url. In Python I sometimes do the following to get the finalurl (not fool-proof): finalUrl = urllib.urlopen("http://www.tinyurl.com/5w5wod").geturl()
- ajkirwin 18y ago
  I added this. A bunch of people demanded it. :) As it stands, it has a few things that twitturly doesn't have, but also they have things I don't. But, it's a beta product and hopefully soon, I can get a slicehost/linode server and get it off of dreamhost and I can spruce it up and add more features.
braindead_in 18y ago
view more information gives an warning. Warning: mysql_fetch_array(): supplied argument is not a valid MySQL result resource in /home/.dodi/jamesgarrett/malicelabs.com/linkalyzer/get_urlstats3.php on line 7 more warnings follow
- joshsharp 18y ago
  Also getting various MySQL warnings.
- ajkirwin 18y ago
  That was my bad! I added some fields to a table and forgot that'd crap all over my UNION.
- wesley 18y ago
  Always always always turn off error display_errors on a production server :)
nirmal 18y ago
Also, every image in your TwitPic section appears once as a png and again as a jpg. Some of these are broken but they still link back to the same TwitPic page.
physcab 18y ago
Nice project. It would be interesting to see maybe a "popularity tree". Like for example, it would be fun to see if Scoble or Kawasaki put out the link first and see how it was distributed. Or if someone unknown sent out the link and someone popular picked it up. Not sure if its possible to trace though. Also, I just started a topic on a data mining related note. If you have any thoughts, I'd love to read them. http://news.ycombinator.com/item?id=406705 http://news.ycombinator.com/item?id=406705
wesley 18y ago
Nice little project, good for learning the API. You may want to check out http://twitturly.com/ http://twitturly.com/ if you haven't yet.
jackowayed 18y ago
Twitpic Hot 30 gives is missing about half of the images. There's spaces for them, but they just show the alt text.

19 points by ajkirwin 18y ago

gsmaverick 18y ago

Neat analyzer! I think there is so much data that can be gathered from Twitter that we have no idea exists.

riklomas 18y ago

Maybe try and reverse engineer the TinyURLs so that users can see where they link to?

iamdave 18y ago
This. Or maybe find a way to grab the page <title> from the target URL and create a separate column with the appropriate information. Otherwise, nifty little app.

nirmal 18y ago

Would be good to expand the tinyurls or at least provide the domain they are pointing too. This may even help you consolidate tinyurls if multiple services are used for the same original url. In Python I sometimes do the following to get the finalurl (not fool-proof): finalUrl = urllib.urlopen("http://www.tinyurl.com/5w5wod").geturl()

ajkirwin 18y ago
I added this. A bunch of people demanded it. :) As it stands, it has a few things that twitturly doesn't have, but also they have things I don't. But, it's a beta product and hopefully soon, I can get a slicehost/linode server and get it off of dreamhost and I can spruce it up and add more features.

braindead_in 18y ago

view more information gives an warning. Warning: mysql_fetch_array(): supplied argument is not a valid MySQL result resource in /home/.dodi/jamesgarrett/malicelabs.com/linkalyzer/get_urlstats3.php on line 7 more warnings follow

joshsharp 18y ago
Also getting various MySQL warnings.
ajkirwin 18y ago
That was my bad! I added some fields to a table and forgot that'd crap all over my UNION.
wesley 18y ago
Always always always turn off error display_errors on a production server :)

nirmal 18y ago

Also, every image in your TwitPic section appears once as a png and again as a jpg. Some of these are broken but they still link back to the same TwitPic page.

physcab 18y ago

Nice project. It would be interesting to see maybe a "popularity tree". Like for example, it would be fun to see if Scoble or Kawasaki put out the link first and see how it was distributed. Or if someone unknown sent out the link and someone popular picked it up. Not sure if its possible to trace though. Also, I just started a topic on a data mining related note. If you have any thoughts, I'd love to read them. http://news.ycombinator.com/item?id=406705 http://news.ycombinator.com/item?id=406705

wesley 18y ago

Nice little project, good for learning the API. You may want to check out http://twitturly.com/ http://twitturly.com/ if you haven't yet.

jackowayed 18y ago

Twitpic Hot 30 gives is missing about half of the images. There's spaces for them, but they just show the alt text.