The fractal behaviour is an undesirable property, not the goal :P ideally every network would be trainable (would converge)! this is the graphed result of a hyp

The fractal behaviour is an undesirable property, not the goal :P ideally every network would be trainable (would converge)! this is the graphed result of a hyperparameter search, a form of optimization in neural networks. If you envision a given architecture as a class of (higher-order) function, the inputs would be the parameters, and the constants would be the hyperparameters. Varying the constants moves to a different function in the class, or, varying the hyperparameters gives a different model with the same architecture (even with the same data).