Using bigger supercomputers to get marginal improvements probably involves overcoming whatever problems exist in training ever larger neural networks, and big n

Using bigger supercomputers to get marginal improvements probably involves overcoming whatever problems exist in training ever larger neural networks, and big nets are inevitably needed for classification of more than just 1000 classes. But the original breakthrough result by the SuperVision team from Toronto (15% error, not completely comparable) trained their net on just 2 GPUs for a week. It would be a pity if it's no long possible to beat the state of the art with your home PC and good ideas.