2 ms·
Ignoring the dimensionality issue for a moment, wouldn't this also represent neural network training on a single training example/batch rather than on the whole
by xg15 3y ago
Ignoring the dimensionality issue for a moment, wouldn't this also represent neural network training on a single training example/batch rather than on the whole trainset?
Still a beginner in this area, but my understanding is that the standard training procedure for neural networks used today involves computing the loss ("height" of the ball) and gradient ("slope" below the ball) for one instance, then moving the ball one step in the gradient's direction, then repeating the process with a different instance.
(You will also typically group a small number of instances together in a batch and process them in the same iteration - but that's for efficiency reasons, not because it's essential to the train algorithm)
So if you wanted to visualise the entire training procedure, I'd imagine an animation of a smoothly rolling ball but with a rapidly changing "landscape" underneath.
Would that be correct?
(Writing this out of genuine curiosity, not as a criticism of the project. I think a lot more work should be done on useful visualisations in this area - and this is an incredibly helpful start.)