3 ms·
Can you talk about how you implemented search-as-you-type? Doing so with semantic search seems tricked given the roundtrips needed to compute embeddings on the
by pjot 2y ago
Can you talk about how you implemented search-as-you-type? Doing so with semantic search seems tricked given the roundtrips needed to compute embeddings on the fly (assuming the use of OpenAI embeddings)
- yujonglee 2y agosure - implementing a search-as-you-type experience with an ai-powered feature was what i wanted to do as well. it doesn't use embeddings at the moment. when you type a short query like 'openai,' it simply runs a basic query using Typesense. however, if you enter a question-like query, such as 'how to llimit api cost,' it transforms it into multiple queries, like 'budget' and 'limit.' in the self-hosted version, it use the CHAT_COMPLETION_MODEL env variable for selecting the llm model. in our cloud version, we use a fine-tuned version of 4o-mini that we will eventually move to a smaller model like llama8b or even 1b.
- pjot 2y agoGot it! I saw this in the code and assumed you were using embeddings def evaluate(input: shared.EvaluationInput): ds = Dataset.from_list(input.dataset) metrics = [metric_map[metric] for metric in input.metrics] llm = ChatOpenAI( model_name=shared.LANGUAGE_MODEL, base_url=os.environ["OPENAI_API_BASE"], api_key=os.environ["OPENAI_API_KEY"], ) embeddings = OpenAIEmbeddings( model=shared.EMBEDDING_MODEL, base_url=os.environ["OPENAI_API_BASE"], api_key=os.environ["OPENAI_API_KEY"], )
- yujonglee 2y agothat piece of code is for llm response evaluation, but we are not really using it at the moment.