3 ms·
> Care to give some examples In the 2010s they were the leader in NLP and the precursor of LLMs like GPT3/3.5/4/4o Machine Translation with Human Parity (2018
by alephnerd 2y ago
> Care to give some examples
In the 2010s they were the leader in NLP and the precursor of LLMs like GPT3/3.5/4/4o
Machine Translation with Human Parity (2018) - https://arxiv.org/abs/1803.05567 https://arxiv.org/abs/1803.05567
MT-DNN (2019) - https://arxiv.org/abs/1901.11504 https://arxiv.org/abs/1901.11504
MASS (2019) - https://arxiv.org/abs/1905.02450 https://arxiv.org/abs/1905.02450
VALL-E (2023) - https://arxiv.org/abs/2301.02111 https://arxiv.org/abs/2301.02111
VALL-E 2 (2024) - https://arxiv.org/abs/2406.05370 https://arxiv.org/abs/2406.05370
While OpenAI was the first to monetize an LLM at scale via ChatGPT, it's still the early stages of this field, and there is a lot of innovation that can still be leveraged, especially in non-English language modeling, machine translation, text-to-speech, etc.
It's in this segment that Microsoft Research shines moreso than even Google Research let alone other organizations because of their strong NLP background in Chinese (Microsoft Research Asia), South Asian languages (Microsoft Research India), Arabic (Microsoft Research's older work during the Iraq War), etc.