The folk theorem of statistical computing [1] says that when you have computational problems, often there’s a problem with your model. So chances are that if yo

The folk theorem of statistical computing [1] says that when you have computational problems, often there’s a problem with your model. So chances are that if your gradient vanishes with 32 bit floats, switching to 64 bit floats won't help much. [1] http://andrewgelman.com/2008/05/13/the_folk_theore/ http://andrewgelman.com/2008/05/13/the_folk_theore/