Those curves of "embedding displacement" are very interesting! quickly scanning the blog led to this notebook which shows how they're computed and shows other

Those curves of "embedding displacement" are very interesting! quickly scanning the blog led to this notebook which shows how they're computed and shows other examples too with similar behavior. https://github.com/spather/transformer-experiments/blob/master/nbs/analyses/70_embedding_adjustments.ipynb https://github.com/spather/transformer-experiments/blob/mast...