3 ms·

How often do papers like this make it to industry applications/published research. Seems stuck in between the two.

• mithametacs 2y ago
  This is going to shred when it reaches industry. But yeah very math heavy for a software engineering paper.
  • ynniv 2y ago
    How long can a page of python take? https://github.com/Zyphra/tree_attention/blob/main/tree_shard_test.py https://github.com/Zyphra/tree_attention/blob/main/tree_shar...
    • mithametacs 2y ago
      You seem to know your stuff. Will this technique work with existing model weights?
      • kasmura 2y ago
        Yes, it is just a way of computing the self-attention in a distributed way
• mjburgess 2y ago
  It's from industry. It's a funded research startup.