You can, and some people will do exactly that. Ideally, new versions of GPT (and other LLMs) will be also trained on the newly added, mostly hallucinated data

You can, and some people will do exactly that. Ideally, new versions of GPT (and other LLMs) will be also trained on the newly added, mostly hallucinated data when using Wikipedia as part of the training set, baking those past hallucinations into the model, allowing other users to get additional hallucinated facts that fit the existing fictitious data, slowly creating a new "GPT cinematic universe" for e.g. small towns in Nebraska. And if people fact check it with Bing, there is a good chance it will say "Yeah, that checks out, here's Wikipedia and <some Wikipedia mirror> as a source". A creative derivation of XKCDs citogenesis[1] :D [1] https://xkcd.com/978/ https://xkcd.com/978/