The overriding bit in terms of why symbolic systems found language challenging, is language in our brains isn't a symbolic system. It has symbolic characteristi

The overriding bit in terms of why symbolic systems found language challenging, is language in our brains isn't a symbolic system. It has symbolic characteristics but is still an analog system inductively learned by neurons. So it is not surprising that models that use approximated neural type information processing, are better able to model something that is naturally implemented with that category of computation. And the reverse is true. Neural models, like humans, don't naturally perform well on many step exact calculations, much less complex exact reasoning. (Unless models are reduced to modules that are equivalent to primitive symbolic operations. But that is essentially symbolic design on top of a substrate whose details have been intentionally abstracted away, with the extra costs of using an arbitrary non-optimal substrate.)