3 ms·
IMO these sorts of methods become interesting the day they are demonstrated working in a conventional AI framework on a commercially important graph like Resnet
by varelse 5y ago
IMO these sorts of methods become interesting the day they are demonstrated working in a conventional AI framework on a commercially important graph like Resnet or BERT. Everything else follows from a killer demo like that.
The benefit is obvious if it works, but it also entails embracing non-deterministic results by design because it's analog. It's also a major shock to the tech stack. It took a half decade for GPUs to be noticed. This is a much more traumatic transition, no?
- version_five 5y agoYeah, I just came here to see if there was any discussion about this. I end up implementing stuff out of research papers pretty much exclusively. If I could train more efficiently, I would happily. But if I have to build something from scratch using an unconventional paradigm with no large body of research behind it, the value to me is negative, because before I even start I would have to convince my customers they can rely on it, that it matches state of the art, etc. Then I'd have to work with a much smaller set of tools built around it, including pretrained models with ultimately millions of dollars of R&D tied up in them. It's good to see new research, but this is not at all immediately helpful for applied work. Show me ResNet implemented this way, with a comparison of speedup and benchmarks on imagenet, and my opinion will change entirely.