There has been some work related to what you are describing: Learning to learn by gradient descent by gradient descent https://arxiv.org/abs/1606.04474 https:/

There has been some work related to what you are describing: Learning to learn by gradient descent by gradient descent https://arxiv.org/abs/1606.04474 https://arxiv.org/abs/1606.04474 Related (but less so), there are also some papers about learning neural network architectures: Designing Neural Network Architectures using Reinforcement Learning https://arxiv.org/abs/1611.02167 https://arxiv.org/abs/1611.02167 Neural Architecture Search with Reinforcement Learning https://arxiv.org/abs/1611.01578 https://arxiv.org/abs/1611.01578