3 ms·
Depends on the task. For code, ask it to implement a not-difficult but not-trivial feature. "Please add hooks to the AnkiDroid source code so that addons would
by dotancohen 3y ago
Depends on the task. For code, ask it to implement a not-difficult but not-trivial feature. "Please add hooks to the AnkiDroid source code so that addons would be viable" might be a good start, for something that is on my mind. Then compare implementations.
For checking hallucinations, ask it about events and trivia that happened eons ago, and also within the last decade. Try some things that it can not possibly know, like how much celery Brad Pitt likes in his salad.
- rajko_rad 3y agoThis is an emerging space with lots of interesting tools coming out... There are many established benchmarks out there (i.e. included on front page of llama2 release), but most product builders have their own sets of evals that are more relevant to them... Here is a thread exploring differences between llama-v2 vs. gpt3.5: https://twitter.com/rajko_rad/status/1681344850510376960 https://twitter.com/rajko_rad/status/1681344850510376960