Be sure to check out 3Blue1Brown's linear algebra series as well. (Maybe after you've built your own MNIST network) Blew my mind when I made the connection that

Be sure to check out 3Blue1Brown's linear algebra series as well. (Maybe after you've built your own MNIST network) Blew my mind when I made the connection that each layer in a dense NN is learning how to do a linear transformation + a non-linear "activation" function.