>> We withhold 120 of the 150 flowers as a test set, and train on 30 flowers, 10 from each species. We designate 5 input nodes (one for each measurement plus on

>> We withhold 120 of the 150 flowers as a test set, and train on 30 flowers, 10 from each species. We designate 5 input nodes (one for each measurement plus one fixed ground) and three output nodes (Fig. 3D inset). Between training steps, the entire test set of 120 flowers is run through the network, and a flower is considered correctly classified if its three out- puts are closest (L2 norm) to the desired outputs of the correct species. I know this is the done thing in machine learning and I don't hold it against the authors, I just hold it up as an example of the broken training regime that is the golden standard throughout machine learning: a test set is said to be "held out" only because it is not used to directly adjust the parameters of a model. In truth, it is not held out, because it is used to control when training has achieved its goal and the inner loop can stop adjusting the model's parameters. In the paper, the device is nominally trained on "30 flowers" but in practice "the entire test set of 120 flowers is run through the network" between training steps. In other words, there is an "inner" learning loop performed on the "30 flowers" but this inner loop is dominated by an outer loop on another 120 flowers. At the end of each inner-outer loop pair, the system has seen all the data. Training ends only once the performance on the 120 flowers has stopped changing. Then it's asked to classify the 120 flowers again. What a surprise, it performs very well. But we have learned nothing of its ability to classify _truly_ unseen data that was not used in either its inner or outer training loop. So from that point of view, I'm sorry to say (and with apologies to the authors who are probably monitoring this thread) but nothing new has been done. The same old bad practices have been encoded in hardware.