xlnet output seemed pretty much I would expect from reader the paper and Transformer-XL examples. I am sad that that it didn't seem to take in bidirectional con

xlnet output seemed pretty much I would expect from reader the paper and Transformer-XL examples. I am sad that that it didn't seem to take in bidirectional context clues, because this a nice differentiation between it and other auto regressive models like GPT-2.