2 ms·
From a sequence modelling perspective it was common to train models for language translation (long before LLMs) on the assumption that, over a long text, the
by PaulHoule 2y ago
From a sequence modelling perspective it was common to train models for language translation (long before LLMs) on the assumption that, over a long text, the cursor in one document follows the cursor in another document. There seems to be a limited amount of variation between grammars of different languages [1] but one of those variations is that you can basically reverse the words in a sentence.
[1] see https://en.wikipedia.org/wiki/Universal_grammar https://en.wikipedia.org/wiki/Universal_grammar