3 ms·

What's specific to deepseek here that other models do not use, or are you just riding the keyword wave?

• t55 2y ago
  DeepSeep proposed the multi-head latent attention technique! :) As far as I know, they are the only ones using it so far
  • karolist 2y ago
    Fair point, thanks for clarification, it seems this was first proposed in https://arxiv.org/pdf/2405.04434 https://arxiv.org/pdf/2405.04434? I was confused by your title mentioning DeepSeek but then first paragraph revert to "...language models like ChatGPT and DeepSeek faster at generating text".
    • t55 2y ago
      Right, that's a good point. I'll adjust the intro a bit. We wanted to provide a more holistic overview on what MLA is, what came before it, and why it matters :) hope it was useful!
      • t55 2y ago
        Just refined it a bit; I hope it's clearer now!