Dedicated hardware isn't magic because there's an intrinsic cost that cannot be reduced. In the case of neural nets you have to store the weights somewhere and

Dedicated hardware isn't magic because there's an intrinsic cost that cannot be reduced. In the case of neural nets you have to store the weights somewhere and you have to perform the multiplications.