3 ms·

Also, did he mean the entire revision history of Wikipedia, or just the most recent versions of pages?

• giancarlostoro 9y ago
  That would definitely explain 2TB haha but yeah, if you pull the recent English version without images it's like ... I wanna say 30GB off the top of my head. As for anyone curious: https://dumps.wikimedia.org/backup-index.html https://dumps.wikimedia.org/backup-index.html
  • 3131s 9y ago
    The most recent was 13.9GB compressed and ~65GB uncompressed. The version without images, talk pages, and revision history is 'enwiki-20180320-pages-articles.xml.bz2' here: https://dumps.wikimedia.org/enwiki/20180320/ https://dumps.wikimedia.org/enwiki/20180320/ The article text contains Wikipedia markup which is a bit difficult to remove but not impossible, there are some existing projects for doing that. DBPedia would have the raw text, but it's not nearly as current.
    • lacksconfidence 9y ago
      You can get the raw text with wikitext and whatnot removed from the search dumps made every week (also has various other metadata) http://dumps.wikimedia.your.org/other/cirrussearch/ http://dumps.wikimedia.your.org/other/cirrussearch/ These are structured with a JSON string for each doc roughly like https://en.m.wikipedia.org/wiki/Foobar?action=cirrusdump https://en.m.wikipedia.org/wiki/Foobar?action=cirrusdump