5 ms·
Language, music, and now amino acid sequences. Attention is all you need.
by lucidrains 7y ago
Language, music, and now amino acid sequences. Attention is all you need.
- mfatica 7y agoI would say you also need a fair bit of data too...
- bearmcbearsly 7y agoWell, yes. But I think lucidrains was referring to: https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762
- return1 7y agoand transformers
- nl 7y agoThe Attention Is All You Need paper is where Transforms were introduced: We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.
- return1 7y agoYup, and the state of the art BERT and gpt-2 are both based on transformers.