Indeed, but I quote: "The pre-training uses stochastic binary units. After pre-training we cheat and use backpropagation by pretending that they are determinist

Indeed, but I quote: "The pre-training uses stochastic binary units. After pre-training we cheat and use backpropagation by pretending that they are deterministic units that send the REAL-VALUED OUTPUTS of logistics." So without trying to sound like a pedant, the manner in which the error is introduced is effectively a hyper parameter, no? The skills involved in doing this correctly will land you the big bucks these days: http://techcrunch.com/2014/01/26/google-deepmind/ http://techcrunch.com/2014/01/26/google-deepmind/ My problem is that I think the programming experience of an entire hardware stack based on 7-bit floating point would be abysmal and that companies that tried to port their software stack to it would die horribly (to be fair they'd deserve it though). OTOH If they'd just stick to the limited domains where this sort of thing is applicable, my perception of it would flip 180 degrees - it's probably a pretty cool image preprocessor. What it's not is the replacement for traditional CPUs. But the people behind this seem to have utter contempt for the engineers tasked with programming their magical doohickey, or putting this in TLDR terms: 1. Scads of 7-bit floating point processors 2. ??? 3. PROFIT!!!