4 ms·
Mind you, this is an optimised version of BERT, from what I could see in their blog post.
by sk0g 6y ago
Mind you, this is an optimised version of BERT, from what I could see in their blog post.
- shubuZ 6y agoThe network topology is the same, no modules are removed, etc. Of course it will be optimized to run on Ampere
- sk0g 6y agoI'll link the the mention of the optimised versions, but that's not what I mean! Say earlier model XYZ trained 4 epochs per hour, and BERT trained 2 epochs per hour. Now on a single card if you can train optimised BERT for 4 epochs per hour, that doesn't necessarily mean the same card will handle XYZ at 8 epochs per hour. It's a technological achievement nonetheless, but the fact that it was heavily optimised for the new architecture, possibly beyond an extent infeasible for non-Nvidia developers, still has to be considered. [0] https://nvidianews.nvidia.com/news/nvidia-achieves-breakthroughs-in-language-understandingto-enable-real-time-conversational-ai https://nvidianews.nvidia.com/news/nvidia-achieves-breakthro...