4 ms·

Can you explain the implications of this for people who aren't expert in the field?

• adamnemecek 4y ago
  I argue that Hopf algebras, which are spicy tensors, provide a better autodiff and are a better model for SSMs, convnets, transformers, and diffusion models.
  • ilaksh 4y ago
    How do we use them in place of transformers? Would they allow something like the power of ChatGPT but with interpretability or modularity? Would they make it easier to create systems that behave like hybrid architectures or automatically combine different types of models (algebras?) It sounds so promising. But I have to assume that if it really is then this can be translated to code that can run on Colab or something, at least by some genius. Until it is I have to be a little skeptical and I'm not going to be able to access it anyway.
    • adamnemecek 4y ago
      Because it would show that all the models are just slightly different Hopf algebras. And yes to interpretability and modularity. It would provide guidance for building new architectures.