noaflaherty

5 ms·

1. 1.▲
   by noaflaherty 4y ago

   Super interesting idea! We already expose UIs and APIs for supplying feedback on the quality of the output, so this could totally be possible once enough feedback has been collected. Thanks for sharing
2. 2.▲
   by noaflaherty 4y ago

   Thanks so much for your interest and thoughtful questions! I'll do my best to answer here, but looking forward to digging in deeper with you offline, too :) A. Documents are uploaded and organized via the UI at this moment, but later t
3. 3.▲
   by noaflaherty 4y ago

   Thank you for the kind words :)
4. 4.▲
   by noaflaherty 4y ago

   Thanks for your interest! We've toyed with the idea of making at least pieces of Vellum open source, but have decided against it for the time being. There are some great open source libraries like Langchain or GPT-Index that, while qui
5. 5.▲
   by noaflaherty 4y ago

   Appreciate the feedback! A comparison table is a great idea and something we'll look into. We fully anticipate having tighter integrations with Langchain in the near future. We view them as complimentary frameworks in many ways. For ex
6. 6.▲
   by noaflaherty 4y ago

   Thanks! We totally agree that spot-checking won't scale long term. We're currently testing a feature in beta that allows you to provide an "expected output" and then choose from a variety of comparison metrics (e.g. exac
7. 7.▲
   by noaflaherty 4y ago

   Thank you for flagging! We've come across them in prior searches, but interesting to learn how well-known they are
8. 8.▲
   by noaflaherty 4y ago

   Thanks for the question! Would you mind elaborating on what you mean by "optimization options?" We've helped a number of our customers fine tune models and optimize for increased quality, lower cost, or decreased latency (e.g
9. 9.▲
   by noaflaherty 4y ago

   We'd love to learn more about what types of guarantees your customers expect – it's likely we can provide many of them now and will inevitably offer even more down the line. Feel free to reach out directly to noa@vellum.ai if you&
10. 10.▲
    by noaflaherty 4y ago

    Hi yes, that's the idea! The example shown in the demo video uses internal help docs as the "source of knowledge" for embeddings, but the same principles apply to customer data.
11. 11.▲
    by noaflaherty 4y ago

    Thank you and good question! If you're comfortable with the quality of OpenAI's embeddings, performing your own chunking, rolling your own integration with a vector db, and don't need Vellum's other features that surroun
12. 12.▲
    by noaflaherty 4y ago

    I appreciate your interest! We'll be reaching out soon :)
13. 13.▲
    by noaflaherty 4y ago

    Thank you for the thoughtful questions! 1) We believe that timing is a critical piece of this opportunity. With the recent media buzz around ChatGPT, we have found that leadership in companies large and small are actively considering how to
14. 14.▲
    by noaflaherty 4y ago

    Great question! We're starting with the manually triggered unit tests in Playground and back-tests prior to updating existing deployments that you see in the demo video, but absolutely envision automated tests being a natural extension
15. 15.▲
    Launch HN: Vellum (YC W23) – Dev Platform for LLM Apps

    136 points by noaflaherty 4y ago|40 comments