3 ms·
What amazed me most was that they used hundreds of millions of unlabelled protein scans. This means we can collect massive data in a new modality, besides the u
by visarga 6y ago
What amazed me most was that they used hundreds of millions of unlabelled protein scans. This means we can collect massive data in a new modality, besides the usual suspects: images, video, audio, text, lidar and sensors. Soon I expect neural implant data to be massive as well.
They surely did unsupervised training on raw data and then fine-tuning on the 170K labelled sequences. I expect the data volume could be increased by orders of magnitude in the next couple of years and we'll see a GPT-3 like jump.