The LLMs don't perform accurate math, particularly floating point, so I'm not sure how they going to be improving their own architectures, since the neural nets

The LLMs don't perform accurate math, particularly floating point, so I'm not sure how they going to be improving their own architectures, since the neural nets rely on weights, not textual tokens. These models are all designed, fine-tuned, adjusted and fed data by humans. You act as if AIs are independent agents in the that have goals to modify themselves. Human engineers are doing that. The models don't have any independent goals. They just respond to prompts.