3 ms·
Wow this is really cool! How did you guys get such big speedups?
by ajaysaini235 5y ago
Wow this is really cool! How did you guys get such big speedups?
- varunkmohan 5y agoThere's some secret sauce behind it, but mostly just using relatively inexpensive cloud inference hardware very effectively. It turns out most of the common NLP frameworks leave a good deal of performance on the table, not to mention the importance of minimizing cloud costs through general methods like using spot instances.