3 ms·
Well, in the transformer forward pass there are a bunch of 4-dimensional arrays being used.
by thatguysaguy 3y ago
Well, in the transformer forward pass there are a bunch of 4-dimensional arrays being used.
- deleted 3y ago[deleted]
- smilekzs 3y agoCame in to say this. The Einsum notation makes it desirable to formulate your model/layer as multi-dimensional arrays connected by (loosely) named axes, without worrying too much about breaking it down to primitives yourself. Once you get used to it, the terseness is liberating.