3 ms·
Stupid question but I had to ask, let us say instead of training anything, you start off with a completely different idea. You take a node, fluctuate its weight
by vivzkestrel 1y ago
Stupid question but I had to ask, let us say instead of training anything, you start off with a completely different idea. You take a node, fluctuate its weights from -0.1 to 0.1, then you add another node with the same thing, then you add a 100 more nodes, 1000 more, another layer of 1000 more, and then you take the first math problem, tweak weights to get the right answer, take the second math problem, tweak weights to get the right answer, do this a billion times, maybe a trillion. will you eventually reach end up making a GPT?
- warsheep 1y agoWhat you're describing is a simplified version of gradient descent (tweaking the weights) and online learning (working on one sample at a time). This version will not get you far, you will just train a model that solves the last math problem you gave it and maybe some others, but it will probably forget the first ones. There are other similar procedures that train better, but they've been tried and are currently worse than classical SGD with large batches
- vivzkestrel 1y agowhat books or courses do you recommend for me to go from the basic neural network to whatever is currently considered cutting edge or atleast standard in terms of AI