To give a concrete example of ChatGPT's shortcomings, I asked it to adjust a pasta recipe to use 50% more pasta, and in addition to increasing the amount of sal

To give a concrete example of ChatGPT's shortcomings, I asked it to adjust a pasta recipe to use 50% more pasta, and in addition to increasing the amount of salt, garlic, etc, it increased the amount of time expected for the pasta to reach al dente from a full boil. It knew to increase the quantities, but it wasn't able to apply commonsense and realize pasta will reach al dente in the same amount of time at a full boil regardless of the quantity. I do think ChatGPT's descendents might become capable of deeper reasoning. A better use of ChatGPT might be to digest and summarize all the documents and then allow a human investigator to interrogate it to more quickly grasp the information and make their own inferences. It's an interesting question as to how trustworthy a summary can be from a system that lacks basic reasoning ability. You can ameliorate the problem to some extent by using ChatGPT to find primary sources to verify key assertions.