There's a few answers/hypotheses already, and complete papers could be written to answer OP's questions, based on which improvements to LLMs could be made to do

There's a few answers/hypotheses already, and complete papers could be written to answer OP's questions, based on which improvements to LLMs could be made to do well on this particular problem class.