5 ms·
I'm pleasantly surprised by how well it works! No offense to Barret Zoph, Quoc Le et al but the "let's throw 10000 GPUs and TPUs at the problem" approach was fr
by JustFinishedBSG 8y ago
I'm pleasantly surprised by how well it works! No offense to Barret Zoph, Quoc Le et al but the "let's throw 10000 GPUs and TPUs at the problem" approach was frankly absurd.
And a huge +1 for the inclusion of random architectures in the results.
- osipov 8y agoWhy do you think that throwing compute at the problem is absurd? Arguably that's the approach that produced the resurgence of interest in CNNs.
- smaddox 8y agoI'm not the original commenter, but I think it's fair to say that throwing compute at the problem is absurd from an academic point of view (where the goal is understanding), but not from a commercial point of view. Although there's still some academic value in throwing compute at a problem, just to see what's possible.
- chadmeister 8y agoYes I think the whole point of Zoph and Le's research is to empirically demonstrate the desired trade off of compute cycles vs feature engineering
- NhanH 8y agoCan you link the paper of Zoph,Le that did the “1000 GPUs” thing? Thanks! I tried searching for it but apparently there was quite a few papers on parameter search.
- kripke 8y agoNot GP, but you are probably looking for the "Neural Architecture Search" series [1] [2] [3]. First one uses something like 1k GPUs for a month, next one is a bit more reasonable, and the last one actually has a reasonable training time, and has comparable performance to DARTS (see ENAS in comparison tables). 1: https://arxiv.org/abs/1611.01578 https://arxiv.org/abs/1611.01578 2: https://arxiv.org/abs/1707.07012 https://arxiv.org/abs/1707.07012 3: https://arxiv.org/abs/1802.03268 https://arxiv.org/abs/1802.03268