The full dataset is available as HTML, XML and SQL from http://download.wikipedia.org/ http://download.wikipedia.org/ . Some versions are updated weekly. Some v

The full dataset is available as HTML, XML and SQL from http://download.wikipedia.org/ http://download.wikipedia.org/ . Some versions are updated weekly. Some versions are updated less frequently. It doesn't occur on a strict schedule because the volume of data increases and sometimes the snapshot process fails. For general use, the HTML version would be easiest to process. The HTML is compressed at a 14:1 ratio using 7zip, so it vastly reduces Wikipedia's server load and bandwidth utilisation. For diffing edit history, the SQL version would be most appropriate because it mirrors the Wikipedia schema, including previous edits. We were using the HTML dataset for testing our own search, but we've outgrown it.