I haven't dug into this much at all, but I am aware that some teams have been looking for methods to either discover or train in symmetries and invariances into

I haven't dug into this much at all, but I am aware that some teams have been looking for methods to either discover or train in symmetries and invariances into latent spaces. For instance, in an image recognition model one might expect shift and rotation invariance properties. I don't know if this line of research is going anywhere, but I thought it was interesting in terms of actual geometry being applied to ANNs.