4 ms·
One area that I would dive into (if I had more time) is "geometric deep learning". i.e) How to design models in a principled way to respect known symmetries in
by angusturner 3y ago
One area that I would dive into (if I had more time) is "geometric deep learning". i.e) How to design models in a principled way to respect known symmetries in the data. ConvNets are the famous/obvious example for their translation equivariance, but there are many recent examples that extend the same logic to other symmetry groups. And then there is also a question of whether certain symmetries can be discovered or identified automatically.
- mrdmnd 3y agoI've been doing some reading on LLMs for protein/RNA structure prediction and I think there's a decent amount of work on SO3 invariant transformer architectures now
- mjhay 3y agoThere's also been some work on more general Lie-group equivariant transformer models. http://proceedings.mlr.press/v139/hutchinson21a/hutchinson21a.pdf http://proceedings.mlr.press/v139/hutchinson21a/hutchinson21...