4 ms·
Do you need an LLM to do this? How much do a word or language model is necessary to do the unscrambling? Could you simply train a raw network on a bunch of scr
by johnsimer 3y ago
Do you need an LLM to do this? How much do a word or language model is necessary to do the unscrambling?
Could you simply train a raw network on a bunch of scrambled and unscrambled text pairs that are representative of the English language? Something with a few hundred million parameters or less?
- johnsimer 3y agoOr at the very least would you have to finetune some 3B or 7B model to do this? I want to create some locally hosted model that can do this in real time with a low memory footprint Let people mash the keyboard at 2x-3x their typing speed and the unscramble it for them in real time to enable typing at 200-300wpm
- arnaudsm 3y agoContextual autocorrect has been on phones for a decade. It's still far from perfect. Many ambiguities are not trivial, even for an LLM
- MacsHeadroom 3y agoMistral-7B does this reliably with no special finetuning, while being 1000x smaller than GPT-4.