3 ms·
Why would you have such a great title and waste it on something dumb like this? You could have written about how language models will pathologically fail any k
by Der_Einzige 3y ago
Why would you have such a great title and waste it on something dumb like this?
You could have written about how language models will pathologically fail any kind of query which requests unique phonetic properties about the output text.
For example, Anthropic's Haiku model (and all other models) cannot write proper Haiku's at all. It's remarkable when it does match the 5-7-5 syllable structure.
You could have written a whole article about that you know. It's even got some neat peer reviewed research about it: https://paperswithcode.com/paper/most-language-models-can-be-poets-too-an-ai-1 https://paperswithcode.com/paper/most-language-models-can-be...
- aftbit 3y ago"Give me ten words that have X as their second letter"
- HarHarVeryFunny 3y agoThat particular shortcoming is more about training data than fundamental limitations of the architecture (of which there are plenty too). To do well on tasks related to pronunciation the model needs either to have been trained on audio data (just starting to be done - e.g. Reka.ai), and/or trained extensively on dictionary/etc pronunciation data and tasks related to its use.