3 ms·
(author here) I actually did a v0 writeup in Go, but I wanted a language that had a bit more powerful type system and more support for a fluent/functional styl
by nelhage 5y ago
(author here)
I actually did a v0 writeup in Go, but I wanted a language that had a bit more powerful type system and more support for a fluent/functional style in some of the expressions. I was optimizing for what I know and felt was highly expressive; I've since gotten a bunch of feedback from people who are interested in the content but not comfortable reading Rust, so perhaps it wasn't the best choice.
- mrfusion 5y agoPython would be awesome!
- mrfusion 5y agoYou should mention that you’re using rust. I got distracted trying I figure out what language you were using.
- ShamelessC 4y agoWhat does the`.0` calling convention mean in Rust? e.g. ``` for (i, r) in right.0.iter().enumerate() { out.0[i] += r; } ``` Aside from that, I thought it was fairly legible. Great write-up by the way. Squashing things into state helps get rid of some of the spookiness created by matrix multiplication and back-propagation. I also really appreciated seeing the explanation on the actual MLP part of the transformer as that is typically assumed to be prior knowledge in other tutorials.
- heinrichf 4y agoIt's to access the tuple's single element, but the author could have used #[repr(transparent)]