10 ms·

Launch HN: Talc AI (YC S23) – Test Sets for AI

132 points by maxrmk 3y ago

Hey all! Max and Matt here from Talc AI. We do automated QA for anything built on top of an LLM. Check out our demo: https://talc.ai/demo https://talc.ai/demo We’ve found that it's very difficult to know how well LLM applications (and especially RAG systems) are going to work in the wild. Many companies tackle this by having developers or contractors run tests manually. It’s a slow process that holds back development, and often results in unexpected behavior when the application ships. We’ve dealt with similar problems before; Max was a staff engineer working on systematic technical solutions for privacy problems at facebook, and Matt worked on ML ops on facebooks’ election integrity team, helping run classifiers that handled trillions of data points. We learned that even the best predictive systems need to be deeply understood and trusted to be useful to product teams, and set out to build the same understanding in AI. To solve this, we take ideas from academia on how to benchmark the general capabilities of language models, and apply them to generating domain specific test cases that run against your actual prompts and code. Consider an analogy: If you’re a lawyer, we don’t need to be lawyers to open up a legal textbook and test your knowledge of the content. Similarly if you’re building a legal AI application, we don’t need to build your application to come up with an effective set of tests that can benchmark your performance. To make this more concrete - when you pick a topic in the demo, we grab the associated wikipedia page and extract a bunch of facts from it using a classic NLP technique called “named entity recognition”. For example if you picked FreeBASIC, we might extract the following line from it: Source of truth: "IDEs specifically made for FreeBASIC include FBide and FbEdit,[5] while more graphical options include WinFBE Suite and VisualFBEditor." This line is our source of truth. We then use an LLM to work backwards from this fact into a question and answer: Question: "What programming language are the IDEs WinFBE Suite and FbEdit designed to support?" Reference Answer: "FreeBasic" We can then evaluate accurately by comparing the reference answer and the original source of truth– this is how we generate “simple” questions in the demo. In production we’re building this same functionality on our customers' knowledge base instead of wikipedia. We then employ a few different strategies to generate questions – these range from simple factual questions like “how much does the 2024 chevy tahoe cost”, to complex questions like “What would a mechanic have to do to fix the recall on my 2018 Golf?” These questions are based on facts extracted from your knowledge base and real customer examples. This testing and grading process is fast – it’s driven by a mixture of LLMs and traditional algorithms, and can turn around in minutes. Our business model is pretty simple - we charge for each test created. If you opt to use our grading product as well we charge for each example graded against the test. We’re excited to hear what the HN community thinks – please let us know in the comments if you have any feedback, questions or concerns!

sherlock_h 3y ago
Looks interesting. How do you rate the correctness? Some complex LLM answers seemed to be correct but not in as much detail as the expected answer. How do you generate the answers? Does the model have access to the original source of truth (like in RAG apps)? And in your examples what model do you actually use?
- maxrmk 3y ago
  Great questions here! > How do you rate the correctness? Some complex LLM answers seemed to be correct but not in as much detail as the expected answer. We support two different modes: a strict pass/fail where an answer has to have all of the information we expect, and a rubric based mode where answers are bucketed into things like "partially correct" or "wrong but helpful". To be honest, we also get the grading wrong sometimes. If you see anything egregious please email me the topic you used at max@talc.ai > How do you generate the answers? Does the model have access to the original source of truth (like in RAG apps)? You guessed right here - we connect to the knowledge base like a RAG app. We also use this to generate the questions -- think of it like reading questions out of a textbook to quiz someong. > And in your examples what model do you actually use? We use multiple models for the question generation, and are still evaluating what works best. For the demo, we are "quizzing" openai's 3.5 turbo model.
tikkun 3y ago
I like the chevy tahoe callback - I'm assuming that's a reference to the chevy dealership that used an LLM and had people doing prompt tricks to get the chatbot to offer them a chevy tahoe for $1. The specificity in your writing above "to make this more concrete" about how it works was also helpful for understanding the product.
- maxrmk 3y ago
  You're exactly right about the chevy tahoe reference. I wasn't sure if anyone would get it. I liked that post a lot, because as much as I think LLMs are going to be useful they have limitations that we haven't solved yet.
3y ago

132 points by maxrmk 3y ago

sherlock_h 3y ago

Looks interesting. How do you rate the correctness? Some complex LLM answers seemed to be correct but not in as much detail as the expected answer. How do you generate the answers? Does the model have access to the original source of truth (like in RAG apps)? And in your examples what model do you actually use?

maxrmk 3y ago
Great questions here! > How do you rate the correctness? Some complex LLM answers seemed to be correct but not in as much detail as the expected answer. We support two different modes: a strict pass/fail where an answer has to have all of the information we expect, and a rubric based mode where answers are bucketed into things like "partially correct" or "wrong but helpful". To be honest, we also get the grading wrong sometimes. If you see anything egregious please email me the topic you used at max@talc.ai > How do you generate the answers? Does the model have access to the original source of truth (like in RAG apps)? You guessed right here - we connect to the knowledge base like a RAG app. We also use this to generate the questions -- think of it like reading questions out of a textbook to quiz someong. > And in your examples what model do you actually use? We use multiple models for the question generation, and are still evaluating what works best. For the demo, we are "quizzing" openai's 3.5 turbo model.

tikkun 3y ago

I like the chevy tahoe callback - I'm assuming that's a reference to the chevy dealership that used an LLM and had people doing prompt tricks to get the chatbot to offer them a chevy tahoe for $1. The specificity in your writing above "to make this more concrete" about how it works was also helpful for understanding the product.

maxrmk 3y ago
You're exactly right about the chevy tahoe reference. I wasn't sure if anyone would get it. I liked that post a lot, because as much as I think LLMs are going to be useful they have limitations that we haven't solved yet.

3y ago