> Learning a transformer language model on such data simply learns a distribution over sequences of tokens. If statistical distributions can model higher leve

> Learning a transformer language model on such data simply learns a distribution over sequences of tokens. If statistical distributions can model higher level polypeptide structure, then it could be useful.