It seems that there are some popular attention methods such as relative embeddings and rotary embeddings (rope embeddings?) that are not possible to implement u

It seems that there are some popular attention methods such as relative embeddings and rotary embeddings (rope embeddings?) that are not possible to implement using pytorch's implementation, if I understand correctly. Do these then require the "slow path" versions that can be more easily modified?