3 ms·
The actual paper can be found at: https://www.iab.cl/wp-content/uploads/2023/11/SSRN-id4573321.pdf https://www.iab.cl/wp-content/uploads/2023/11/SSRN-id4573321
by araes 3y ago
The actual paper can be found at: https://www.iab.cl/wp-content/uploads/2023/11/SSRN-id4573321.pdf https://www.iab.cl/wp-content/uploads/2023/11/SSRN-id4573321...
Note: After reading the paper, there are serious methodology issues. ChatGPT without any human involvement (simply reading the task as instructions and producing an answer) produced a better "quality" result than any human involvement in many tasks.
My read: ChatGPT produces what we pre-view as "correct".
They even kind of state this: "when human subjects use ChatGPT there is a reduction in the variation in the eventual ideas they produce. This result is perhaps surprising one would assume that ChatGPT, with its expansive knowledge base, would instead be able to produce many very distinct ideas, compared to human subjects alone."
My read: The humans converge towards what is already viewed as "correct". Its like being in a business and your boss already knows exactly what your boss actually wants, and any variation you produce is automatically "bad" anyways.
There is also a drastic difference in the treatment of the in/out/on "frontier" subject. Lots of talk about how great adding AI is to "inside frontier" tasks, no similar graphs/charts for "outside frontier" tasks. Finally, if these people are consultants and business profs, they produce some crazily bad charts. Figure 3 in the appendix is so difficult to read.