4 ms·
In typical ML you just learn the weights. But the architecture stays fixed, and since we don't have an architecture that can realistically solve well enough al
by Voloskaya 8y ago
In typical ML you just learn the weights.
But the architecture stays fixed, and since we don't have an architecture that can realistically solve well enough all tasks, it shouldn't count as Godel machine.
Meta-learning, where you also learn hyper-parameters (including potentially the architecture) are closer to that, see for example:
- Google Vizier: https://ai.google/research/pubs/pub46180 https://ai.google/research/pubs/pub46180
- Microsoft neural architecture search: https://github.com/Microsoft/nni https://github.com/Microsoft/nni
But this is still far from a true Godel machine.
- jimfleming 8y agoHyperparameter optimization (including architectures) is not really meta-learning. Meta-learning, also known as "learning to learn", is more like MAML[0], RL2[1], L2RL[2], etc. 0. https://arxiv.org/abs/1703.03400 https://arxiv.org/abs/1703.03400 1. https://arxiv.org/abs/1611.02779 https://arxiv.org/abs/1611.02779 2. https://arxiv.org/abs/1611.05763 https://arxiv.org/abs/1611.05763