4 ms·
> "The cost of training is the actual bottleneck in AI," This really isn't true in general. Maybe in select subfields of AI like language model pertaining this
by jdeaton 6y ago
> "The cost of training is the actual bottleneck in AI,"
This really isn't true in general. Maybe in select subfields of AI like language model pertaining this is true.
- canjobear 6y agoWhat other bottlenecks do you have in mind?
- RhysU 6y agoResearcher time: Data munging, apples-to-apples model evaluation, identifying and fixing broken assumptions only detectable due to subtle model behavior.
- plaidfuji 6y agoSure, but I assumed he was referring to the biggest computational bottleneck. All of those others are just tooling problems or data quality problems. For DNNs the hours-to-weeks-long training times make it hard to iterate manually or do any kind of rational optimization of architecture and hyper parameters.
- RhysU 6y agoSure, if we just had stationary, perfect data and perfect objective functions it's only computational, operational, monitoring, and maintenance complexity that holds us back.