4 ms·

How long can a page of python take? https://github.com/Zyphra/tree_attention/blob/main/tree_shard_test.py https://github.com/Zyphra/tree_attention/blob/main/tre

by ynniv 2y ago

How long can a page of python take? https://github.com/Zyphra/tree_attention/blob/main/tree_shard_test.py https://github.com/Zyphra/tree_attention/blob/main/tree_shar...

mithametacs 2y ago
You seem to know your stuff. Will this technique work with existing model weights?
- kasmura 2y ago
  Yes, it is just a way of computing the self-attention in a distributed way