Do LLMs need to be good at math with the same approach? To draw an a analogy, we've got our human brain specialized. Why not implement a part of the AI brain

Do LLMs need to be good at math with the same approach? To draw an a analogy, we've got our human brain specialized. Why not implement a part of the AI brain that's not neural nets, but instead circuitry specialized to math? Maybe a dumb question since I'm a layperson!