4 ms·
There's more detailed examination of Transformers from circuit complexity in this paper: https://arxiv.org/abs/2106.16213 https://arxiv.org/abs/2106.16213 Ther
by shawntan 3y ago
There's more detailed examination of Transformers from circuit complexity in this paper: https://arxiv.org/abs/2106.16213 https://arxiv.org/abs/2106.16213
There are a few references also in the blog post.
- krackers 3y agoThank you! I'd love more articles analyzing LLMs from a circuit-complexity perspective! Also seems modeling the transformer as in the TC0 class, in principle it should be able to do division and multiplication of numbers.