10 ms·
Could there be some value in recognizing areas where the model needs finer grained weights and somehow using a different data type just in certain areas?
by bilsbie 3y ago
Could there be some value in recognizing areas where the model needs finer grained weights and somehow using a different data type just in certain areas?
- fabiospampinato 3y agoIt seems tough to do, besides I'm not sure what the benefit would be, with that you can't do the optimized matrix multiplication anymore, and if you need more precision presumably you can just add more neurons and/or train for longer and/or with better data.