Yes you can easily train with xlnet instead of roberta-base --- just write the different model name in the config file (or pass a different string value, if doi

Yes you can easily train with xlnet instead of roberta-base --- just write the different model name in the config file (or pass a different string value, if doing it from Python). You can find an example config file here: https://github.com/explosion/projects/blob/v3/benchmarks/ner_conll03/configs/transformer.cfg https://github.com/explosion/projects/blob/v3/benchmarks/ner... I actually didn't see a performance improvement when using XLNet over roberta-base. I always wondered about this; ages ago I looked into it and I wasn't sure that the preprocessing details in the transformers version were entirely correct. Given very similar accuracies from XLNet and RoBERTa, I preferred RoBERTa for the following reasons: * I've never been able to understand the XLNet paper :(. I spent some time trying when it was released, but I just didn't really get it, not anything close to the level where I'd be able to implement it, anyway. * Standardising on BERT architecture has some advantages. If we mostly use BERT, we have a better chance of using faster implementations. Mostly nobody is training new XLNet models, whereas many new BERT models are being trained.