I'd like to believe that GPU's and cloud might allow for more scientific exploration of the "hows" of learning via many small experiments gradually revealing li

I'd like to believe that GPU's and cloud might allow for more scientific exploration of the "hows" of learning via many small experiments gradually revealing limitations and characteristics until finally insight. Using high speed hardware can allow someone to do 10's or scores of runs a day. If you are doing one every 2 weeks or so then it's really, really hard to make any progress at all because you daren't take risks. So the productivity of 80 a day vs 2 per month isn't just 100x it's lots and lots more. Also as you say it's lowered the bar which means that teams can onboard grad students and interns and get them to do something that's useful - it may be trivial - but it's useful.