I have been asking myself the question of how useful an LLM would be that has perfect intelligence. Meaning that for any question you give it that can possibly

I have been asking myself the question of how useful an LLM would be that has perfect intelligence. Meaning that for any question you give it that can possibly be answered with the given information, you will get the right answer. Obviously, it would be very useful, but still limited by it's context and prompt. For many tasks, coding models are getting close. It will do everything I ask generally correctly the first time. Around half the re-dos are because I under-specified the prompt. Soon that will be 100% of re-dos, and the programming aspect of my job will be mostly focused on writing good prompts, yet I will still be here identifying and translating real-world requirements into prompts. We are quickly approaching a situation where LLMs can ace all benchmarks, and yet still not see the insane ROI that the frontier labs are predicting because humans are a bottleneck, and so is experiment. For example that perfect LLM may be able to find the cure to cancer, but all the research in the world isn't enough information to answer that question, so we need to conduct experiments and learn more. Maybe that can speed up humanity's cancer discoveries by 10X, but not 1000X, purely because of the experiment bottleneck.