4 ms·
That's because the sample size is probably small and for niche prompts or topics. It's very hard to evaluate whether a model is better than another, especially
by imperio59 2y ago
That's because the sample size is probably small and for niche prompts or topics.
It's very hard to evaluate whether a model is better than another, especially doing it in a scientifically sound way is time consuming and hard.
This is why I find these types of comments like "model X is so much better than model Y" to be about as useful as "chocolate ice cream is so much better than vanilla"
- r2_pilot 2y agoAnd both flavors have a base flavor of excrement... Still, since I started using Claude 3 Opus (and now 3.5 Sonnet) a couple of months back, I don't see myself switching from them nor stopping use of LLM-based AI tech; it's just made me feel like the computer is actually working for and with me and even that alone can be enough to get me motivated and accomplish what I set out to do.
- skapadia 2y ago"it's just made me feel like the computer is actually working for and with me and even that alone can be enough to get me motivated and accomplish what I set out to do." This is a great way to describe what I've been feeling / experiencing as well.
- r2_pilot 2y agoJust an update on my initial impressions of Claude 3.5 Sonnet. It's a better programmer than I am in Python; that's not saying much, but this is now two nights in a row I've been impressed with what I've created with it.
- stavros 2y agoTrue, I just tried it for generating a book summary, and Sonnet 3.5 was very bad. GPT-4o is equally bad at that , gpt-4-turbo is great.
- netsec_burn 2y agoThis more likely has to do with context length?
- stavros 2y agoNo, all the information is there, but gpt-4o tends to produce bullet points (https://www.thesummarist.net/summary/the-making-of-a-manager/your-first-three-months/ https://www.thesummarist.net/summary/the-making-of-a-manager...), whereas gpt-4-turbo tends to produce much more readable prose (https://www.thesummarist.net/summary/supercommunicators/the-matching-principle-how-to-fail-at-recruiting-spies/ https://www.thesummarist.net/summary/supercommunicators/the-...).
- Obscurity4340 2y agoHow is prose more readable than bullets?
- stavros 2y ago* Clearer narrative * Connection between points * Flows better * Eyes don't start-stop as much
- Obscurity4340 2y agoI think I was thinking more along the lines of if I'm looking for specific information or to get a condensed understanding of something Different readable than the more flowing, conjunct readable than yours (which is the more typical use of it I concede)