3 ms·

Traveling words: a geometric interpretation of transformers

• dleeftink 3y ago
  I like this definition. Simple, clean, effective. Maybe the start of a new field tackling 'travelling wordsman' (smith?) problems.
• dpflan 3y ago
  To anyone knowledgeable: where does geometric deep learning fit in here? Is this paper just another geometric view, or is it an attempt at a formalization for transformer mathematics? I don't see prominent GDL authors in this paper's references (Bronstein, Cohen, Bruna, Veličković, ...).
  • esafak 3y ago
    Geometric learning is more about group- and invariant theory. This paper only goes as far as linear algebra, as far as I see.
  • uoaei 3y ago
    I think it could be applicable to developing a novel geometric theory if the dynamics of the word trajectories were better understood.