You tune it first. A better faith version of this should have been the experimenters creating a creative writing pipeline they found created the best output an

You tune it first. A better faith version of this should have been the experimenters creating a creative writing pipeline they found created the best output and then testing that. Using naive zero shot prompts in a single pass with no CoT doesn't even reflect real world production usage outside of the casual user of ChatGPT who discovered it a week ago. So yes, surprise surprise - casual and naive use of a tool isn't comparable to experts without the tool. And I'm sure my using a hammer to build a deck would perform much worse than an expert carpenter without a hammer. But that's saying less about the utility of the hammer than it is my ability to use the hammer.