I agree on f32, although I still think the differential approach will remain dominant for large systems. Formally, that's likely because of training efficiency

I agree on f32, although I still think the differential approach will remain dominant for large systems. Formally, that's likely because of training efficiency of gradient-based approaches. More speculatively, I think beliefs in general benefit from having associated values (strongly believing something, versus weakly believing something) which to me suggest an inherent benefit of numeric values inside the networks (see fuzzy logic[1]). Of course, sometimes you're dealing with rules that are less gradual, s.t. boolean logic (or maybe something like fp4) may suffice. To me the non-differential approach will shine on very constrained environments and open quite a few applications. In the near future we might have very efficient and effective code generation from LLMs which changes the landscape as well.