Now, what is the point in trying to generate any kind of "meaning" from those documents if each document consists of completely meaningless gibberish? A lot of

Now, what is the point in trying to generate any kind of "meaning" from those documents if each document consists of completely meaningless gibberish? A lot of data with a small amount of noise can be much more informative than a small amount of "perfect" data. If 1-5% of your examples are noise, you can still extract meaning, and find patterns. Also, that's not meaningless gibberish, there is still signal in those examples.