6 ms·

I'm curious what technology you used to grab content from wikipedia?

• andre 19y ago
  it's pretty easy to write a scraper in most languages and customize it as needed. I've done in PHP.
  • spoonyg 19y ago
    Easy, but really if a site is nice enough to provide a legitimate way for you to get data you should respect that and download it.
• amichail 19y ago
  You can download it here: http://download.wikimedia.org http://download.wikimedia.org
• dazzawazza 19y ago
  Wikipedia provides a query api for accessing pages and links within pages: http://meta.wikimedia.org/wiki/Query http://meta.wikimedia.org/wiki/Query Its very useful and I think people should use it if they want to scrape Wikipedia. At least it gives the Wikipedia guys a chance to throttle bandwidth if they need to.