That's when learning rate changed to a smaller number. The graph mainly shows that with different initialization scheme, the network starts descending initially

That's when learning rate changed to a smaller number. The graph mainly shows that with different initialization scheme, the network starts descending initially faster.