3 ms·
Generalized Language Models
- ArtWomb 8y agoJust came to post this link as well. Progress in machine translation has truly been amazing. Also check out the recent work from Google on Transformer-XL https://ai.googleblog.com/2019/01/transformer-xl-unleashing-potential-of.html https://ai.googleblog.com/2019/01/transformer-xl-unleashing-... And for deep background, these recent sets of notes from Stanford's Deep Learning NLP class: http://web.stanford.edu/class/cs224n/ http://web.stanford.edu/class/cs224n/ https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_TcylNr7EkRe6 https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_...
- yoquan 8y agoHmm, the article's summary table showed "only the top task layer(s)" for BERT fine-tuning. Is that correct? The original paper [1] wrote: "All of the parameters of BERT and W are fine-tuned jointly to maximize the log-probability of the correct label." [1]: https://arxiv.org/abs/1810.04805 https://arxiv.org/abs/1810.04805
- lilianweng 8y agothanks! my text should be correct but the table is wrong. just fixed.