3 ms·
Instead of using someone else's language library pre-trained on a bunch of text, you can create your own language library trained on your own text. Use cases i
by _akhe 3y ago
Instead of using someone else's language library pre-trained on a bunch of text, you can create your own language library trained on your own text.
Use cases include: Auto-completion, auto-correct, spell checking, search/lookup, conversation simulation (chatbot), and more.
"Trained" means that every word of every sentence of every book loaded in has an embedded score representing how likely it is to appear following a user-provided cursor. The more text you have trained on (see /training/embeddings/) the more scored words it has available, so the more robust and accurate it can be with language.
What would make your implementation a "Large" Language Model is if it has many, many, many words and phrases modeled and ranked, not just a small amount.
Chat Bots:
For something like a realistic conversation simulator (AKA a chat bot) you usually need a Large Language Model (LLM) so that any user-provided cursor yields relevant responses. To filter out the noise of less relevant responses, a good LLM-based chatbot will also use parts-of-speech and sentiment analysis ("attention") to match to the most relevant one.
Example cursor: "Paris is", you might get "in France" or "really nice" as equally ranked predictions, what would make a Chat-specific model choose one or the other would be analyzing the input - example if the question was "Where is Paris?" the Chat model would be biased toward answers that mention locations, and respond with "in France".
User: "Where is Paris?"
Convert to cursor: "Paris is"
Get completions: "in France", "really nice", "a first name", etc.
Analyze input: "Where is" refers to a location
Analyze completions: "in France" refers to a location
Answer: "Paris is in France"
Hope this helps!