3 ms·
Has anyone looked through all the responses and chosen any winners?
by sixQuarks 3y ago
Has anyone looked through all the responses and chosen any winners?
- rubymamis 3y agoGPT4 seems to me to be the best. Undi95/ReMM-SLERP-L2-13B the runner up.
- obiefernandez 3y agoWhat a name!!
- rootusrootus 3y agoClearly the prompt construction and initial instructions are critically important here. Without that, the ReMM-SLERP-L2-13B model produces awful results. Blank answers about half the time. For lay users, ChatGPT remains the undisputed winner. Although I do see reasonably good results with the more recent llama-2 70B variations, which are plausibly useful a majority of the time.
- adrian_b 3y agoI have not looked at all responses, but I have found interesting that at the question about the sisters of Sally, initially all LLMs have provided wrong answers. Nevertheless, when the initial prompt has been augmented with "Let's think step by step.", that helped 2 LLMs to give the right answer: GPT 4 and ReMM SLERP L2 13B. So at least for this kind of problem, there were 2 clear winners. The fact that one of them is GPT 4, is not a surprise.
- smilingemoji 3y agoI took at the look at the SVG smilies, only GPT gets it right (https://benchmarks.llmonitor.com/svg https://benchmarks.llmonitor.com/svg). You can preview the SVGs drawn by the LLMs by pasting this into the browser console document.querySelectorAll("td pre").forEach((node) => { let code = node.textContent; node.insertAdjacentHTML('afterend', code) }) Or take a look at my screenshot: https://i.ibb.co/Kw0kp58/Screenshot-2023-09-09-at-17-15-20-https-benchmarks-llmonitor-com.png https://i.ibb.co/Kw0kp58/Screenshot-2023-09-09-at-17-15-20-h...
- ComputerGuru 3y agoClever use of the dev tools console. I think you’re being a little uncharitable, the prompt said “smiley” and not “smiling emoji” and “smiley” was once upon a time used for the whole slew of emoji. With that in mind, several others also pass.
- dr_kiszonka 3y agoGood point, but you may be too charitable. At least according to Wikipedia, a smiley is supposed to be smiling. https://en.wikipedia.org/wiki/Smiley https://en.wikipedia.org/wiki/Smiley
- ComputerGuru 3y agoI don’t know about Wikipedia but I’m speaking from lived historical experience. Here’s a throwback for you: https://www.phpbb.com/customise/db/styles/smilies-13 https://www.phpbb.com/customise/db/styles/smilies-13 (All “emoji” images were just called smilies in a lot of software.)
- vincelt 3y agoOh neat, thanks for sharing, wanted to add an interpreter to that test