I would like to understand the role of quality vs quantity in these models. Is it better to train them on slight variations of what humans consider to be high q

I would like to understand the role of quality vs quantity in these models. Is it better to train them on slight variations of what humans consider to be high quality input or just add a boatload of low quality input instead? If we somehow found an original Shakespeare book, wouldn't it be drowned by all the trashy Amazon erotica written every year? I understand that there are reasons to expect improvement with a broader set of inputs - the model would never understand slang and other key language components by being trained only in academic papers. I wonder whether the Shakespeare example would be worked out by it somply occupyinf a novel high-dimensional space because of its uniqueness, or whether there is a signal to noise issue here.