5 ms·
Processing half float data on the CPU will be tricky without hardware support. There's plenty of CPU processing needed for deep learning, for capturing and prep
by modeless 12y ago
Processing half float data on the CPU will be tricky without hardware support. There's plenty of CPU processing needed for deep learning, for capturing and preparing datasets.
Half precision floating point is really something that should have been in hardware a long time ago. There are other applications besides deep learning that could benefit from the dynamic range of floating point, but don't need 32 bits. For example, imaging and audio.
- bitL 12y agoFor both imaging and audio even 64 bit doubles aren't sufficient if you use many filters together.
- raverbashing 12y ago[Citation needed] It might be not enough if you have a lot of filters and doesn't want to care about their ordering/gains etc If you have something like 16bit exponent and 48bit mantissa (64-bit) that's absurd
- listic 12y agoIt looks like that's what F16C instruction set was made for http://en.wikipedia.org/wiki/F16C http://en.wikipedia.org/wiki/F16C
- Narishma 12y agoThat looks to be just for converting to and from half-precision floats.
- minthd 12y agoLogarithmic number systems could offer even greater improvement(100x over gpu), if you're willing to live with more errors and/or invest some time in fixing them: http://www.gwern.net/docs/2010-bates.pdf http://www.gwern.net/docs/2010-bates.pdf
- DennisP 12y agoSeems like a great idea, and the last couple pages sound like they were making good progress toward commercializing it. But the document's from 2010. I wonder if they're still working on it.
- minthd 12y agoI know it was implemented in some military UAV, and achieved great results. So maybe the military took this. But it still doesn't mean it can't be implemented outside the u.s. or in FPGA's. Even a simulation of this in software could be really interesting.