5 ms·

A Mathematical Perspective on Transformers

• ottaborra 3y ago
  Beautiful piece of math. Similar work: Traveling Words: A Geometric Interpretation of Transformers https://arxiv.org/abs/2309.07315 https://arxiv.org/abs/2309.07315
  • 3abiton 3y ago
    That was an interesting abstract, looking forward to read it!
    • tudorw 3y ago
      Would this kind of understanding aid in extracting 'skills' from an LLM by identifying the relevant topology and isolating that part? Then we could have a toolbox of skills to assemble into what we needed?
• MichaelRazum 3y ago
  Maybe the right title would be a (statistical) physics perspective...
• SpaceNoodled 3y ago
  Disappointed that this wasn't about robots in disguise.
  • p1esk 3y ago
    It is.