Interesting, but then how do you explain that rectified linear operations between layers work better than sigmoids? According to your logic, ReLU should have w

Interesting, but then how do you explain that rectified linear operations between layers work better than sigmoids? According to your logic, ReLU should have worse error propagating quality than squashing functions?