One of the major selling points of neural networks is that you can model/learn relationships that are very hard to explicitly write down. Also keep in mind that

One of the major selling points of neural networks is that you can model/learn relationships that are very hard to explicitly write down. Also keep in mind that neural networks often work with floating point values to scale input values. Sure, you can write down these floating point values in a diagram, but the numbers themselves will not tell you why they have the magnitude that they have. With that being said, there is research happening that tries to find better tools to understand the reasoning of neural networks. And sometimes we might actually see and understand a general decision pattern of a NN that we could adopt for a simpler model. But in many cases it will not be possible to come up with a simpler model because some phenomena have an intrinsic complexity.