The bias is often updated/corrected in the back propagation step as well. The purpose of the value is to shift the activation function along the x-axis basicall

The bias is often updated/corrected in the back propagation step as well. The purpose of the value is to shift the activation function along the x-axis basically. While the weights define the slope of the activation function.