dabs

5 ms·

1.
by dabs 2y ago
Seriously, this is a statement you can make in promptfoo configuration: ``` assert: - type: equals value: 'Hello, World!' ``` We are redefining the equality operator in YAML. Someone please explain.
2.
Why is promptfoo so popular?(github.com)
4 points by dabs 2y ago|3 comments
3.
by dabs 2y ago
I have seen frameworks in other spaces attempt the configuration driven approach and have seen it always fall out of favor vs. programmatic approaches. I am reading the docs right now and I feel like it's so much work to try to learn t
4.
by dabs 2y ago
Please do surfing next and add it to Surfline
5.
by dabs 2y ago
thanks for the list, super handy!
6.
by dabs 2y ago
Testing AI systems is challenging. We ran into this question collaborating on an AI system that revises sales contracts. Some of the questions we had to answer were: - What makes the advice output by an AI assistant "good legal advice&
7.
Making a Legal AI App Unit Testable with Vitest(docs.poyro.dev)
4 points by dabs 2y ago|1 comments
8.
by dabs 2y ago
Testing AI systems is challenging. We ran into this question collaborating on an AI system that revises sales contracts. Some of the questions we had to answer were: - What makes the advice output by an AI assistant "good legal advice&
9.
Essay: How to Write Unit Tests for Your AI Web App(docs.poyro.dev)
3 points by dabs 2y ago|0 comments
10.
Essay: AI Testing as Part of Full-Stack Engineering(docs.poyro.dev)
1 points by dabs 2y ago|1 comments
11.
by dabs 2y ago
In chatting with many full-stack engineers, both at start ups and larger companies, I've noticed many are just dipping their toes into testing the AI components of their apps. Many don't know the best ways to go about it or what k
12.
Show HN: Poyro – Simple library with Jest-like API for unit testing LLM outputs(docs.poyro.dev)
3 points by dabs 2y ago|1 comments
13.
by dabs 2y ago
Hey all, my friend and I were building a web app that involves calling an LLM and ran into the question of testing. Although there are many great solutions out there, we couldn't find one that checked all of these boxes: - Not configur
14.
by dabs 2y ago
Good question! ChatGPT certainly makes it easier to implement a prototype that works quite well for documents meeting certain conditions. For example, it does pretty well with restaurant menus out of the box because the entities extracted t
15.
Show HN: Papermint – Use a simple description to extract key phrases from PDFs(papermintai.com)
4 points by dabs 2y ago|2 comments
16.
by dabs 8y ago
Question for someone with a more theoretical background: The paper shows that the EMX learnability of some class of functions with respect to some set of probability distributions is undecidable. Does EMX learnability encompass all notions