25 ms·
They do mention the 8b+ "GPT-2" model trained by nVidia. Which is their reference to megatron.
by ivalm 7y ago
They do mention the 8b+ "GPT-2" model trained by nVidia. Which is their reference to megatron.
- lucidrains 7y agoOh! You are right! How did I miss that..