Only skimmed the paper, but I'd say one potential issue is that they show good results in tabular data domains, where it's likely easier to get away with minima

Only skimmed the paper, but I'd say one potential issue is that they show good results in tabular data domains, where it's likely easier to get away with minimal models than in other domains, such as visual/lidar/audio data. And it seems to me those are the domains where it'd actually be more interesting to deploy this kind of model. In either case, there seems to be a tradeoff in that while you're saving space/energy with the hardware model, updating that model after its deployed might be more complicated than with a typical software-only model.