4 ms·
Variable precision sounds scary, but maybe it could work. You should look into building support for your chips into Theano, Torch7, and/or Caffe. You should be
by modeless 11y ago
Variable precision sounds scary, but maybe it could work. You should look into building support for your chips into Theano, Torch7, and/or Caffe. You should be able to hide Unum or any other quirks behind the interfaces of those libraries, so people can drop in their existing models with no work. If you can show a significant training speed advantage over GPUs, you'll have a market.
- trsohmers 11y agoYou should check unum out... It is actually higher accuracy then IEEE float, as it does not have "rounding", which causes errors over time. unum is to floating point as floating point is to integer. The other nice thing is that it is a superset of IEEE float, and has a "IEEE mode" where you can convert to IEEE float, which is also jokingly called the "guess" function. As for support, that is the plan. Right now our customers have been most interested in high end signal processing, so we have been taking time to port FFTW, but supporting Theano/Torch/Caffe, etc is a relatively straightforward process.
- Keyframe 11y agoDidn't Cray had something like that, a variable floating point where you could opt how many bits (up to a limit) you wanted? I vaguely remember hearing about it from a talk Computer History Museum had.
- tim333 11y agoI just stuck a bit on unums on Wikipedia as there didn't seem to be much. https://en.wikipedia.org/wiki/John_Gustafson_(scientist)#Unums https://en.wikipedia.org/wiki/John_Gustafson_(scientist)#Unu... Feel free to correct/improve it anyone.
- trsohmers 11y agoThe End of Error (http://www.amazon.com/The-End-Error-Computing-Computational/dp/1482239868 http://www.amazon.com/The-End-Error-Computing-Computational/...) is the book describing them in great detail... I'm working with John to put together a publicly accessible wiki, which I hope will be up in the next month or two. That being said, the book is worth having.