2 ms·
When people say "oh this LLM is 13B parameters", how many parameters is "y = x * 2"? 1? How long/how many iterations would I have to train a 1 layer NN (and wh
by MuffinFlavored 3y ago
When people say "oh this LLM is 13B parameters", how many parameters is "y = x * 2"? 1?
How long/how many iterations would I have to train a 1 layer NN (and what kind of layers should even be used, it gets so confusing with the activation layers, layer types, etc.) to actually do y = x * 2
https://stats.stackexchange.com/questions/217703/can-a-deep-neural-network-approximate-multiplication-function https://stats.stackexchange.com/questions/217703/can-a-deep-...
- kolinko 3y agox2 or xy? x2 is one neuron and probably a dosen iterations max. xy may be a bit more complicated - like the stackoverflow explains 13B parameters - yes, it’s essentially 13B multiplications/additions (the most basic neuron is y=activation_function(x*weight+bias).