3 ms·
Where is the performance data?
by yeahwhatever10 2y ago
Where is the performance data?
- felarof 2y ago(author here, sorry for the delay in replying, was stuck in back-to-back meetings) I updated our github repo to include GPU, VRAM utilization data (https://github.com/felafax/felafax?tab=readme-ov-file#amd-405b-fine-tuning-run https://github.com/felafax/felafax?tab=readme-ov-file#amd-40...) Note: we couldn't run the JIT-compiled version of the 405B model due to our code/VRAM constraints (we need to investigate this further). The entire training run was executed in JAX eager mode, so there is significant potential for performance improvements. GPU utilization across the board was still ~30-40% even with eager mode, which is quite good! With JIT, I think the GPU util can easily shoot up to ~50-60%.
- dahart 2y agoI’d be interested to hear how long it takes, wall clock, to train the same model to the same loss with same number of batches. I don’t trust utilization to say anything useful.