5 ms·

Comparing a system to MNIST isn't really an indicator that it provides an actual improvement. Comparing it to VGG would be a better indicator.

• p1esk 9y ago
  MNIST is a dataset, VGG is a software model (network architecture). You can train/test VGG on MNIST, train/test this spiking network model on MNIST, and compare the accuracy between the two. If you want to compare energy efficiency, you look at the hardware these models are running on (CPU, GPU, FPGA, ASIC), and see how much energy is spent by each of these chips to reach a certain accuracy (preferably on ImageNet). You can also compare the speed of processing (significantly affected by image size): 1. How fast can you get to a certain accuracy? 2. How many images per second can you classify with that accuracy?
  • 0xbear 9y ago
    >> MNIST is a dataset, VGG is a software model Strictly speaking, true. But one discovers early enough that you can screw up a number of things when training a model on something as simple as MNIST and the model will still work, because there's so much redundancy, the correct layers will just train their way around the ones that don't really work. VGG trained for general purpose image classification with a decent sized input is a much better indicator of whether or not some ML hardware improvement is useful.
    • p1esk 9y ago
      You're contradicting yourself. If you're afraid that redundancy will hide hardware problems, then VGG is a bad choice, because it's pretty much the most redundant NN model ever :)