4 ms·

It feels weird to me to use the hyper parameters as the variables to iterate on, and also wasteful. Surely there must be a family of models that give fractal li

It feels weird to me to use the hyper parameters as the variables to iterate on, and also wasteful. Surely there must be a family of models that give fractal like behaviour ?

amelius 3y ago
> It feels weird to me to use the hyper parameters as the variables to iterate on Yes, I also think this is strange. In regular fractals the x and y coordinates have the same units (roughly speaking), but here this is not the case, so I wonder how they determine the relative scale.
- thfuran 3y ago
  Is there really any meaningful sense in which real and imaginary numbers have the same units but two dimensionless real hyperparameters don't?
  - xvedejas 3y ago
    Complex numbers multiplied by an imaginary number rotate preserving magnitude. In this sense they have the same units.
    - thfuran 3y ago
      You can rotate any vector while preserving its magnitude.
      - xvedejas 3y ago
        If the units are different, you do need to come up with a conversation between them, or you're implicitly saying it's 1-to-1
freeone3000 3y ago
The fractal behaviour is an undesirable property, not the goal :P ideally every network would be trainable (would converge)! this is the graphed result of a hyperparameter search, a form of optimization in neural networks. If you envision a given architecture as a class of (higher-order) function, the inputs would be the parameters, and the constants would be the hyperparameters. Varying the constants moves to a different function in the class, or, varying the hyperparameters gives a different model with the same architecture (even with the same data).