3 ms·
AIUI they (1) finetuned LLaMA on 75B tokens of medical papers, and called the resulting model PMC-LLaMA, (2) finetuned that model on task-specifc data, then (3)
by kristjansson 3y ago
AIUI they (1) finetuned LLaMA on 75B tokens of medical papers, and called the resulting model PMC-LLaMA, (2) finetuned that model on task-specifc data, then (3) separately finetuned the base model they started with in (1) on the same task specific data for comparison. So, they're reporting 6 separate conditions: {LLaMA, PMC-LLaMA} x {Zero-shot, Finetune, PEFT}.
Could definitely use some more exposition around that though.