5 ms·
I've found einsum to be amazing at consolidating my code into something more readable, particularly for implementing architectures from scratch. Here's a good
by imranq 5y ago
I've found einsum to be amazing at consolidating my code into something more readable, particularly for implementing architectures from scratch.
Here's a good video that explains why its so good: https://www.youtube.com/watch?v=pkVwUVEHmfI https://www.youtube.com/watch?v=pkVwUVEHmfI
Also check out Lucid Rains Github, who uses it extensively to build transformer architectures from scratch: https://github.com/lucidrains https://github.com/lucidrains
\
* Example: https://github.com/lucidrains/alphafold2/blob/d59cb1ea536bc5d2e0eee81056d786cb1011ac57/alphafold2_pytorch/alphafold2.py#L150 https://github.com/lucidrains/alphafold2/blob/d59cb1ea536bc5...
- p1esk 5y agoamazing at consolidating my code into something more readable Readable if you're already familiar with einsum notation. Otherwise there's learning curve. An alternative to einsum is using multiple dot product and reshape ops, hopefully with each one having a comment - this would be a lot more readable imo.
- hakuseki 5y agoI really don't find that einsum is less intuitive than reshape. Maybe they both take about the same time to learn the first time. And that's less time than it would take me to struggle through even a single function using reshape ops.
- p1esk 5y agoHow many DL people are familiar with dot products and reshapes, and how many are familiar with einsum?
- hakuseki 5y agoI really couldn't say. It does occur to me, though, that you are probably talking about the elementwise product, not the dot product. The dot product includes a reduce_sum step and outputs a scalar.
- pizza 5y agoBut then to find some mistake you would have to corroborate n * 2 pieces of information, which is quite a bit more, before getting into the issue of whether or not the sequence of operations actually makes sense on top of all that.
- p1esk 5y agoSo let’s say there’s a bug, and an einsum statement is a suspect. Assume I’m not familiar with einsum notation, but I know dot products and reshapes. How do I debug it?
- pizza 5y agoGuess you can’t, then. Personally I’m a bigger fan of einops since the variable/index names can be a lot more self-documenting.