3 ms·
Can we build a model based purely on search? The model searches until it finds an answer, including distance and resolution Search is performed by a DB, the q
by nico 4y ago
Can we build a model based purely on search?
The model searches until it finds an answer, including distance and resolution
Search is performed by a DB, the query then sub-queries LLMs on a tree of embeddings
Each coordinate of an embedding vector is a pair of coordinate and LLM
Like a dynamic dictionary, in which the definition for the word is an LLM trained on the word
Indexes become shortcuts to meanings that we can choose based on case and context
Does this exist already?
- fzliu 4y agoNot sure what you mean by dynamic dictionary, but the embedding tree you mention is already freely available Milvus via the Annoy index.
- nico 4y agoAn entry in a dictionary is static text, ex: per·snick·et·y: placing too much emphasis on trivial or minor details; fussy. "she's very persnickety about her food" A dynamic entry could instead be an LLM what will answer things related to they word, ex: What is the definition of persnickety? How can I use it in a sentence? What are some notable documents that include it? Any famous quotes? … So each entry is an LLM trained mostly only on that keyword/concept definition There are some that believe in smaller models: https://twitter.com/chai_research/status/1655649081035980802?s=46 https://twitter.com/chai_research/status/1655649081035980802...