I don’t think the choice of using floating point numbers is particularly inefficient. If frameworks were fixed point by default it would be tricky to get the dy

I don’t think the choice of using floating point numbers is particularly inefficient. If frameworks were fixed point by default it would be tricky to get the dynamic ranges right all the way through a network. The math in the training assumes the numbers are continuous, too.