Page 9 on the Llama tech report has an interesting graph that predicts task level performance from the cross-entropy loss. The sigmoidal model fits well, and a

Page 9 on the Llama tech report has an interesting graph that predicts task level performance from the cross-entropy loss. The sigmoidal model fits well, and at the steepest part of the S, a .01 change in NLL is worth about 5% task level accuracy. Here is a quick screenshot if you are lazy. https://snipboard.io/C6mipQ.jpg https://snipboard.io/C6mipQ.jpg And here is the paper if you want to dig deep into the highest quality published research on LLM frontier. https://ai.meta.com/research/publications/the-llama-3-herd-of-models/ https://ai.meta.com/research/publications/the-llama-3-herd-o...