Pre-training refers to unsupervised training that's done before a model is fine-tuned. The model still starts out random before it's pre-trained. Here's where

Pre-training refers to unsupervised training that's done before a model is fine-tuned. The model still starts out random before it's pre-trained. Here's where the Othello paper's weights are (randomly) initialized: https://github.com/likenneth/othello_world/blob/master/mingpt/model.py#L127 https://github.com/likenneth/othello_world/blob/master/mingp...