3 ms·

> how it makes me choose the correct number of neurons in a layer, how many layers, the activation function Seeing massive ablation studies on each one of thos

> how it makes me choose the correct number of neurons in a layer, how many layers, the activation function Seeing massive ablation studies on each one of those in just about every ML paper should be fairly indicative that nobody knows shit about fuck when it comes to that. Just people trying things out randomly and seeing what works, copying ideas from each other resulting in some vague guidelines. It's the worst field if you want things to be logical and explainable. It's mostly labelling datasets, paying for compute and hoping for the best.

bob1029 2y ago
> nobody knows shit about fuck when it comes to that This is why I've abandoned neural networks as a computational substrate for genetic programming experiments. Tape-based UTMs may be extremely rigid in how they execute instruction streams, but at least you can eventually understand and describe everything that contributes to their behavior. Changing the fan-out from 12 to 15 in a NN is like ancient voodoo ritual compared to realizing a program tape is probably not long enough based upon rough entropy measures.