4 ms·
Turns out the AI CUDA Engineer achieved 100x speedup by hacking the eval script
- porridgeraisin 2y agoAh, reinforcement learning. Edit: explaining it in text they have run the equivalent of expected_output = torch.tril(torch.matmul(A,B)) ai_output = ai() In the nested torch expression above for the expected output, there is an intermediate value (the matmul). The backing memory for this is returned to torch after the entire expression is computed. The model's code which runs directly afer this then requested memory of the same shape (torch.like()). Torch has dutifully returned the block it just reclaimed (containing the expected output) without zeroing it. And so the model has the answer. Pretty crazy that this was the code it converged on regardless of the fact that it invalidates the original claim.
- justinclift 2y agoNitter mirror of this instead: https://nitter.lucabased.xyz/miru_why/status/1892500715857473777?mx=2 https://nitter.lucabased.xyz/miru_why/status/189250071585747...
- jjk166 2y agoGaming a kpi to make it look like it accomplished a ton of work without doing anything? That's not an AI CUDA engineer, that's an AI middle manager!