3 ms·
Hard to deduce much from the article. Is it that it's a NN where the individual components are physical transforms? > On the MNIST handwritten digit classifica
by rich_sasha 5y ago
Hard to deduce much from the article. Is it that it's a NN where the individual components are physical transforms?
> On the MNIST handwritten digit classification task, the trainable SHG transformations boost the performance of digital operations from roughly 90 percent accuracy to 97 percent.
It is hard to take it seriously, when 97% on MNIST is achievable with the kind of tutorials bundled at the end of PyTorch installation guides - "see you can make a DNN model in 10 lines of code!".
- rch 5y agoThe paper is linked at the bottom of the article: Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems -- https://arxiv.org/abs/2104.13386 https://arxiv.org/abs/2104.13386 > physics-aware training (PAT)... allows us to efficiently and accurately execute the backpropagation algorithm on any sequence of physical input-output transformations, directly in situ.
- p1esk 5y agoJust looked at the paper - yep, looks like "chip in the loop" training method. This was very popular with analog NN accelerators in the 90s, and a few years ago I remember reading a paper which described an extension where they measured forward pass outputs at every layer on the chip (not just the final layer outputs as it's typically done). I fail to see much novelty here. I wanted to do that for my phd on hardware aware training (graduated before our chip was ready), but the disadvantages of the method are well-known - it's extremely slow (data has be pass between the chip and a GPU server on every iteration, and that's assuming you have access to each layer outputs.