4 ms·
This is a cool demo, but note the author doesn’t actually show the rewritten headlines are better in any way but spot-checking. The conclusion here isn’t that G
by goodside 4y ago
This is a cool demo, but note the author doesn’t actually show the rewritten headlines are better in any way but spot-checking. The conclusion here isn’t that GPT-3 can optimize titles on its own, but that it generates ideas that, when reviewed by a human, can be usefully cherry-picked for inspiration. The task of ensuring, e.g., that the headline is factually accurate is left to the human.
- minimaxir 4y agoThere isn't an easy way to prove one headline is better than another outside of A/B testing, unfortunately.
- RosanaAnaDana 4y agoEven then, is there even a way to do so with A/B testing? Even in A/B testing you can't fully control for the environment a headline sits in. Whats happening in the world? What styles of headlines are trending? Proving things is very difficult, and probably not necessary.
- minimaxir 4y ago"Proving", in this case, is whether the performance of one title is better than another at statistically significant level. Performance can be faceted by referrer if necessary if there's a need to control the results further, or exclude sources where title A/B testing can't feasibly be done or has noticeably different behavior. (an example is using a different title for social sharing vs. the article title itself) This is generally how title A/B testing works at every major online publisher.