It is, but in many cases its still pretty easy to figure out. It depends on how complicated the problem is. If it's truly an incomprehensible nonlinear cesspool

It is, but in many cases its still pretty easy to figure out. It depends on how complicated the problem is. If it's truly an incomprehensible nonlinear cesspool, then its pretty much impossible. If it's secretly a linear regression with a few forks and inflection points, its easy. The problems neural nets are used for do tend to fall into the former, but the techniques to unravel it are getting better.