2 ms·
Just looked at the paper - yep, looks like "chip in the loop" training method. This was very popular with analog NN accelerators in the 90s, and a few years ago
by p1esk 5y ago
Just looked at the paper - yep, looks like "chip in the loop" training method. This was very popular with analog NN accelerators in the 90s, and a few years ago I remember reading a paper which described an extension where they measured forward pass outputs at every layer on the chip (not just the final layer outputs as it's typically done).
I fail to see much novelty here. I wanted to do that for my phd on hardware aware training (graduated before our chip was ready), but the disadvantages of the method are well-known - it's extremely slow (data has be pass between the chip and a GPU server on every iteration, and that's assuming you have access to each layer outputs.