3 ms·
Deeper independent, sure. Deeper shared though? Information and signal theory still apply here. At infinite cycles, without new input, you'll end up with a lock
by nomel 1mo ago
Deeper independent, sure. Deeper shared though? Information and signal theory still apply here. At infinite cycles, without new input, you'll end up with a locked state or oscillations. Some point before that, any "attractors" in the latent space, with slightly higher statistics, will pull things towards a space that might eventually be only loosely related to the goal, because each loop would be lossy, right?
- Legend2440 1mo ago>At infinite cycles, without new input, you'll end up with a locked state or oscillations. I don't think that's true; there are computations that take infinite steps but never converge or repeat, like the mandelbrot set. Looping for millions or billions of steps is absolutely normal in traditional algorithms. We know from complexity theory that some computations require a minimum number of steps. More depth is just more room for computation.
- pennomi 1mo agoOr even more simply, calculating the digits of pi is an infinite number of steps.
- nomel 1mo agoSure, but I don't think that's related to what you're implying, which is that, correct me if I'm wrong, the same number of weight should be able to hold many orders of magnitude more information by being reused. These are not deterministic functions or systems that have infinite precision. See the "should I drive or walk my car to the car wash", or any of the other logic riddle problems, for examples of a statistical attractors.
- Legend2440 1mo ago>the same number of weight should be able to hold many orders of magnitude more information by being reused. No, not hold more information, perform longer computations. E.g. if you want to solve sudokus, you will need more and more loops for larger grids. There is no shortcut.