AFAIK the training works by just selecting random words in sentences to have the LLM "fill in" until it generalizes to being able to write entire sentences. The

AFAIK the training works by just selecting random words in sentences to have the LLM "fill in" until it generalizes to being able to write entire sentences. The training data can be completely unstructured.