Some cool examples, like an animated attention head layer: https://bhosmer.github.io/mm/index.html?0=out+%3D+%28attn+%3D+%28Q+%3D+input+%40+wQ%29+%40+%28K+%3D+w

Some cool examples, like an animated attention head layer: https://bhosmer.github.io/mm/index.html?0=out+%3D+%28attn+%3D+%28Q+%3D+input+%40+wQ%29+%40+%28K+%3D+wK+%40+input_t%29%29+%40+%28V+%3D+input+%40+wV%29+%40+wO&1=out&2=none&83=true&84=open&3.1=attn+%40+V&3.4=true&3.5=32&3.6=32&3.7=row+major&3.8=&3.9=-1&3.10=1&3.11=0&3.2=none&12.13=inherit&14.15=1&16.17=positive&16.18=left&16.19=bottom&16.20=back&21.2=softmax%28tril%28x%2Fsqrt%28k%29%29%29&22.13=inherit&23.15=1&24.17=negative&24.18=left&24.19=top&24.20=front&25.2=none&26.13=inherit&27.15=1&28.17=positive&28.18=left&28.19=bottom&28.20=back&29.1=input&29.4=false&29.5=64&29.6=64&29.7=gaussian&29.8=&29.9=-1&29.10=1&29.11=0&29.0=&30.1=wQ&30.4=false&30.5=64&30.6=16&30.7=gaussian&30.8=&30.9=-1&30.10=1&30.11=0&30.0=&25.1=Q&25.4=true&31.2=none&32.13=inherit&33.15=1&34.17=positive&34.18=right&34.19=top&34.20=back&35.1=wK&35.4=false&35.5=16&35.6=64&35.7=gaussian&35.8=&35.9=-1&35.10=1&35.11=0&35.0=&36.1=input_t&36.4=false&36.5=64&36.6=64&36.7=gaussian&36.8=&36.9=-1&36.10=1&36.11=0&36.0=&31.1=K&31.4=true&21.1=attn&21.4=true&37.2=none&38.13=inherit&39.15=1&40.17=negative&40.18=right&40.19=top&40.20=back&41.1=input&41.4=false&41.5=64&41.6=64&41.7=gaussian&41.8=&41.9=-1&41.10=1&41.11=0&41.0=&42.1=wV&42.4=false&42.5=64&42.6=16&42.7=gaussian&42.8=&42.9=-1&42.10=1&42.11=0&42.0=&37.1=V&37.4=true&43.1=wO&43.4=false&43.5=16&43.6=64&43.7=gaussian&43.8=&43.9=-1&43.10=1&43.11=0&43.0=&44.45=none&44.46=20&44.47=false&44.13=vvprod&44.48=0&49.50=1&49.51=1&49.15=1&52.53=blocks&52.54=4&52.55=0&52.56=1&52.57=0&52.17=negative&52.18=left&52.19=top&52.20=front&58.59=6&58.60=true&58.61=2&58.62=0.6&58.63=0.5&58.64=0.5&58.65=10&58.66=false&58.67=false&68.69=local&68.70=0.2&68.71=0.4&68.72=0.7&68.73=1.25&68.74=0.77&68.75=0.74&68.76=0.04&77.8=&78.79=-134.23103479750094&78.80=75.74999372781257&78.81=272.1936120907652&82.79=0&82.80=0&82.81=0&expr=0&name=1&epilog=2&left=3&matmul=4&h=5&w=6&init=7&url=8&min=9&max=10&dropout=11&left.anim=12&alg=13&left.block=14&k+blocks=15&left.layout=16&polarity=17&left+placement=18&right+placement=19&result+placement=20&left.left=21&left.left.anim=22&left.left.block=23&left.left.layout=24&left.left.left=25&left.left.left.anim=26&left.left.left.block=27&left.left.left.layout=28&left.left.left.left=29&left.left.left.right=30&left.left.right=31&left.left.right.anim=32&left.left.right.block=33&left.left.right.layout=34&left.left.right.left=35&left.left.right.right=36&left.right=37&left.right.anim=38&left.right.block=39&left.right.layout=40&left.right.left=41&left.right.right=42&right=43&anim=44&fuse=45&speed=46&hide+inputs=47&spin=48&block=49&i+blocks=50&j+blocks=51&layout=52&scheme=53&gap=54&scatter=55&molecule=56&blast=57&deco=58&legends=59&shape=60&spotlight=61&row+guides=62&flow+guides=63&lens+size=64&magnification=65&interior+spotlight=66&axes=67&viz=68&sensitivity=69&min+size=70&min+light=71&max+light=72&elem+scale=73&zero+hue=74&hue+gap=75&hue+spread=76&diag=77&cam=78&x=79&y=80&z=81&cam.target=82&compress=83&folder=84 https://bhosmer.github.io/mm/index.html?0=out+%3D+%28attn+%3...