Maybe! I'm lost in the maths, "Our results suggest that learned optimizers can benefit from considering the (symmetry) structure of the weight space they optimi

Maybe! I'm lost in the maths, "Our results suggest that learned optimizers can benefit from considering the (symmetry) structure of the weight space they optimize. " this from 7th Feb came out of DeepMind; https://arxiv.org/abs/2402.05232 https://arxiv.org/abs/2402.05232 When it comes to the math underneath an LLM, https://medium.com/autonomous-agents/part-8-mathematical-explanation-of-why-its-hard-for-llms-to-memorize-86ee1d58eeb9 https://medium.com/autonomous-agents/part-8-mathematical-exp... is about the most accessible explanation I have found so far.