3 ms·
You should check out the VOLT paper, I think it would work well. It's a new technique for splitting up a vocabulary into subwords while minimizing entropy. Thes
by rajansaini 5y ago
You should check out the VOLT paper, I think it would work well. It's a new technique for splitting up a vocabulary into subwords while minimizing entropy. These subwords could then be mixed and matched, maybe by a neural model, for better results.
- lioeters 5y agoThank you for the reference. To save others a search, I believe this is the paper: Vocabulary Learning via Optimal Transport for Neural Machine Translation - https://arxiv.org/abs/2012.15671 https://arxiv.org/abs/2012.15671 https://jingjing-nlp.github.io/volt-blog/ https://jingjing-nlp.github.io/volt-blog/ https://github.com/Jingjing-NLP/VOLT https://github.com/Jingjing-NLP/VOLT