The method is straightforward to extend to n-grams, this is actually what they do (see table 1 in https://arxiv.org/abs/1804.07755 https://arxiv.org/abs/1804.0

The method is straightforward to extend to n-grams, this is actually what they do (see table 1 in https://arxiv.org/abs/1804.07755 https://arxiv.org/abs/1804.07755). And since you are simply learning a rotation matrix, there is no risk of overfitting.