4 ms·
This is going to shred when it reaches industry. But yeah very math heavy for a software engineering paper.
by mithametacs 2y ago
This is going to shred when it reaches industry.
But yeah very math heavy for a software engineering paper.
- ynniv 2y agoHow long can a page of python take? https://github.com/Zyphra/tree_attention/blob/main/tree_shard_test.py https://github.com/Zyphra/tree_attention/blob/main/tree_shar...
- mithametacs 2y agoYou seem to know your stuff. Will this technique work with existing model weights?
- kasmura 2y agoYes, it is just a way of computing the self-attention in a distributed way