4 ms·
Tranfer learning is a form of this, where you train the first half of a neural net over a huge set of data, then take a much more specific task and train just a
by gleenn 7y ago
Tranfer learning is a form of this, where you train the first half of a neural net over a huge set of data, then take a much more specific task and train just a bit more by adding some layers, and you get great sometimes industry leading results. Look up Google's BERT project, trained generically on English texts and then you can reuse that part in many different tasks like sentiment analysis or entity recognition or translation. An amazing way to leverage Googles compute and data access and do cool things even as a single developer.
- bitL 7y ago> as a single developer I think you are out of luck already. Even with transfer learning, retraining BERT to fit a larger dataset takes ages even on an RTX 8000. A single state-of-art experiment/training run at Facebook now costs 7 figures. The age of individual SOTA ML researcher/developer is over.
- raducu 7y agoThis is so sad, actually, and It's not just ML, it is every kind of new tech and discovery. To me it shows we have already picked the easy fruits and technology will not continue to evolve exponentially.