ofermend

8 ms·

61.
by ofermend 3y ago
We've updated the Hughes Hallucination Evaluation Model (HHEM) with results about Phi-2. TL;DR: slightly better than Mixtral-8x7B
62.
by ofermend 3y ago
We just updated the Hughes Hallucination Evaluation model (HEM) with the results of using the latest (beta) version of Palm 2. TL;DR - much better than before.
63.
by ofermend 3y ago
Really impressed with the progress of Anthropic with this release. I would love to see how this new version added to Vectara's Hallucination Evaluation Leaderboard. https://huggingface.co/spaces/vectara/Hallu
64.
by ofermend 3y ago
Excited to see GPT4-Turbo and longer sequence lengths from OpenAI. We just released Vectara's "Hallucination Evaluation Model" (aka HEM) today https://huggingface.co/vectara/hallucination_evaluation_mode.
65.
by ofermend 3y ago
Excited about GPT4-Turbo and longer sequence lengths. Looking forward very much for faster inference. We just released Vectara's "Hallucination Evaluation Model" (aka HEM) today https://huggingface.co/vectara&
66.
by ofermend 3y ago
First time I ran into embeddings was with word2vec and could not resist showing that, similar to the "king - man + woman ~ queen", it also the case that "yoda - good + evil ~ vader". It's also cool that the semanti
67.
by ofermend 3y ago
I compared the embedding models of OpenAI, Cohere and Vectara using Llama_Index for an end-to-end RAG question-answering flow. Here are the results.
68.
Ask HN: How important are embedding models to RAG?
2 points by ofermend 3y ago|1 comments
69.
by ofermend 3y ago
Using Boomerang can significantly improve your end-to-end RAG performance: retrieving the most relevant facts (or chunks) matters, a lot!
70.
by ofermend 3y ago
RAG is a very useful flow but I agree the complexity is often overwhelming, esp as you move from a toy example to a real production deployment. It's not just choosing a vector DB (last time I checked there were about 50), managing it,
71.
by ofermend 3y ago
Yes totally agree with that (and other comments below). Moving from a toy example to production deployment requires all the things we are used to having in robust/mature products like postgres.
72.
by ofermend 3y ago
Agreed. Utilizing the power of LLMs with RMs (retrieval models) can be much more powerful, and I expect RAG implementations to progress in that direction in the coming years.
73.
by ofermend 3y ago
I agree. my experience is that hybrid search does provide better results in many cases, and is honestly not as easy to implement as may seem at first. In general, getting search right can be complicated today and the common thinking of &quo
74.
by ofermend 3y ago
Yes, plus we certainly have too many independent vector DB products as it is.
75.
by ofermend 3y ago
Yeah, but so many? I'm counting 47 alone on the LangChain page.
76.
Ask HN: Do we need so many vector databases?
1 points by ofermend 3y ago|2 comments
77.
by ofermend 3y ago
It's not impossible that fine-tuning would also help RAG. but it's certainly not guaranteed and hard to control. Fine-tuning essentially changes the weights of the model, and might result in other, potentially negative outcome, li
78.
by ofermend 3y ago
Totally agree. retrieval augmented generation is still the preferred way to give the LLM more knowledge. Fine-tuning is mostly useful for adapting the base model for another task. I wrote about this in a recent blog post: https:/&#x2F
79.
by ofermend 3y ago
retrieval augmented generation (we call it "grounded generation" at Vectara) is a great way to build GenAI apps with your data. This blog post can be useful: https://vectara.com/a-reference-architecture-for-ground
80.
by ofermend 3y ago
It's important to note that this fine-tuning is what is known as "supervised fine-tuning" where you give the LLM a set of question/answer pairs and it tunes to those (see https://huyenchip.com/2023/0
81.
by ofermend 3y ago
Great blog post from Vectara's Tallat and Talip: explaining the various approaches to vector search.
82.
by ofermend 3y ago
We intend to keep it up as a service to the community. No current plan to take it down. Why?
83.
Show HN: Ask Feynman(askfeynman.demo.vectara.com)
1 points by ofermend 3y ago|3 comments
84.
by ofermend 3y ago
There are so many options for vector databases that it's so confusing. But those are just a piece of the puzzle when you create applications using large language models. As mentioned in the comments, you have to choose an embeddings m
85.
Fine-Tuning or Grounded Generation?(vectara.com)
1 points by ofermend 3y ago|1 comments
86.
by ofermend 3y ago
When applying GenAI to "your data": do you use fine-tuning or grounded generation (aka retrieval augmented generation)?
87.
by ofermend 3y ago
very cool. But like many other uses of LLM it can hallucinate and/or produce a wrong result. For example I tried: "gorilla dry run of brew upgrade" And got a response that didn't work.
88.
Reasons to Use Vectara with LangChain(vectara.com)
2 points by ofermend 3y ago|0 comments
89.
Show HN: Why are there protests on Reddit” – AskNews demo(asknews.demo.vectara.com)
4 points by ofermend 3y ago|1 comments
90.
Show HN: Vectara-Answer(github.com)
2 points by ofermend 3y ago|1 comments

More ›