There are dumps available that already have lots of metadata extracted and the content available as plaintext without wikimarkup or html. Specifically these are

There are dumps available that already have lots of metadata extracted and the content available as plaintext without wikimarkup or html. Specifically these are the dumps of the search indices used at wikimedia: https://dumps.wikimedia.org/other/cirrussearch/current/ https://dumps.wikimedia.org/other/cirrussearch/current/ These dumps contain a json line per article and are formatted roughly like so: https://en.wikipedia.org/wiki/Hacker_News?action=cirrusdump https://en.wikipedia.org/wiki/Hacker_News?action=cirrusdump