3 ms·

That kind of compute still costs money. You still have to find the funding for it.

• sillysaurusx 5y ago
  Nah, a motivated hacker can use a Colab TPU or GPU for free. It's how I started. There's an ungodly amount of resources available now compared to even 10 years ago, let alone 30. The bottleneck is usually motivation.
  • YeGoblynQueenne 5y ago
    Note that all the advances that eventually led to today's successes and applications of deep learning happened 20 or more years ago (e.g. LSTMs was in the '90s, the neocognitron in the '80s and so on). Then the hardware and the data came online and the field boomed, but no progress has been made in terms of new approaches and the overall state of research has stagnated with the inevitable results discussed in the article above (I can quote at least one Turing Award winner on that if that sounds just like me bloviating). In a sense, because it's now easy enough to get good results by throwing a bunch of data at a large computer, everyone's at it and nobody is looking for a way to obtain good results without a lot of data and a big computer. Which of course is not sustainable, not least because it means only large corporations can obtain state of the art results (and to be honest, I don't think any such have been obtained by "motivated hackers" using colab). No, my hope is that things will continue as they are now, until academics realise they've been kicked out of the game by the Big Players, and then go look for a way to compete that doesn't require gigantic amounts of compute. After all, academia can still motivate people to do actual research and find actually new things. The same motivation is much harder in industry, that only cares about one thing and only knows how to hand out one kind of reward.