3 ms·
This doesn't pass the sniff test for me. Not sure if these models are memorizing the answers or something else, but it's simply not the case that they're as cap
by lukev 3y ago
This doesn't pass the sniff test for me. Not sure if these models are memorizing the answers or something else, but it's simply not the case that they're as capable as a domain expert (yet.)
I do not have a PhD, but in areas I do have expertise, you really don't have to push these models that hard to before they start to break down and emit incomplete or wrong analysis.
- p1esk 3y agoHave you tried the Opus model specifically?
- matchagaucho 3y agoThey claim the model was grounded with a 25-shot Chain-of-Thought (CoT) prompt.
- CamperBob2 3y agoThe idea is that they aren't as capable as a domain expert, but they are more capable than an expert from a different domain. E.g., if you ask a chemistry PhD something about genetics or astrophysics, you're more likely to get a correct answer from the model. Which is pretty interesting IMO.