4 ms·

It's public data! You could scrape and index it now for free if you wanted...

• bootload 16y ago
  "... It's public data! You could scrape and index it now for free ..." That is the most insightful thing I've read today.
• petercooper 16y ago
  Though I've noticed only pages up to 200 work when going back through history.. this only gets you a few days back on the most popular tags.
  • mikeklaas 16y ago
    Sure, but user pages go back farther than that...
• scorpion032 16y ago
  They even throttle the friendfeed scraper which graciously pulls all its users data at once. You can't write a simple scraper that is not distributed in 100 of machines across the web to pull out their data.
  • viraptor 16y ago
    I heard that there is this thing called "the cloud" where you can rent services based on the work time. That makes cheapo servers both realistic and quite simple ;) Actually I just noticed you get 750h of free micro instance time from aws... I wonder if it would be worth doing. I imagine the link+tags are <100GB in total.