This is an impressive demonstration of GPT-4's logic and reasoning capabilities, but the DM result uses a real-world reward signal (e.g. the list is sorted corr

This is an impressive demonstration of GPT-4's logic and reasoning capabilities, but the DM result uses a real-world reward signal (e.g. the list is sorted correctly) to validate its results whereas we can never be sure that GPT-4 outputs are hallucinated or not. Also this experiment would never be done if not for the DM paper in the first place.