4 ms·

See the section "Comparison with BERT" which has a nice worked example that is reasonably understandable without necessarily fully understanding the rest of the

• jeremysalwen 7y ago
  That section you are saying was reasonably understandable was the part that confused me. For example: > it is obvious that XLNet always learns more dependency pairs given the same target and contains “denser” effective training signals BERT is only masking 15% of the tokens, so isn't the amount of dependency pairs like 18% higher at most?
  • yorwba 7y ago
    > BERT is only masking 15% of the tokens, so isn't the amount of dependency pairs like 18% higher at most? That's a small but significant difference, allowing them to improve performance by a small but significant amount.