3 ms·Just FYI when you lookup visualizations of the attention matrices, they tend to be somewhat symmetric.by adamnemecek 4y agoJust FYI when you lookup visualizations of the attention matrices, they tend to be somewhat symmetric.