4 ms·

>They note that in practice, most AI researchers will not do that. I think the main issue there is that these kernel compilers are not well-integrated into the

>They note that in practice, most AI researchers will not do that. I think the main issue there is that these kernel compilers are not well-integrated into the ML libraries. If it's a standard part of it, well-documented, easy to build (usually the difficult part is compiling someone's custom compiler...), and if the compiled results can be "stored" for future ML models to just use, I think it would see more adoption. As of now, tensor compilers for ML frameworks are more of a (good but shaky) research tool. I think a Julia approach of "do it on Julia code" (even the compiler, so it's easy to just ]add the package) could definitely break down some of these barriers to getting it done in a way that garners widespread adoption, but a lot of work will need to be done in order to get a good enough tensor compiler for people to care. If anything, this is a fun space with many opportunities.

cs702 7y ago
Clearly, that would help. And clearly, there are opportunities for improvement -- as much as two orders of magnitude (!) in the case of capsule networks. That said, I think the authors are right about the challenge here: "...current frameworks excel at workloads where it makes sense to manually tune the small set of computations used by a particular model or family of models. Unfortunately, frameworks become poorly suited to research, because there is a performance cliff when experimenting with computations that haven’t previously been identified as important. While a few hours of search may be acceptable before production deployment, it is unrealistic to expect researchers to put up with such compilation times (recall this is just one kernel in what may be a large overall computation); and even if optimized kernels were routinely cached locally, it would be a major barrier to disseminating research if anyone who downloaded a model’s source code had to spend hours or days compiling it for their hardware before being able to experiment with it." EDIT: removed third paragraph.

cs702 7y ago

Clearly, that would help. And clearly, there are opportunities for improvement -- as much as two orders of magnitude (!) in the case of capsule networks. That said, I think the authors are right about the challenge here: "...current frameworks excel at workloads where it makes sense to manually tune the small set of computations used by a particular model or family of models. Unfortunately, frameworks become poorly suited to research, because there is a performance cliff when experimenting with computations that haven’t previously been identified as important. While a few hours of search may be acceptable before production deployment, it is unrealistic to expect researchers to put up with such compilation times (recall this is just one kernel in what may be a large overall computation); and even if optimized kernels were routinely cached locally, it would be a major barrier to disseminating research if anyone who downloaded a model’s source code had to spend hours or days compiling it for their hardware before being able to experiment with it." EDIT: removed third paragraph.