2 ms·
Why the amount of the training data for LLM is less than for the distilled and task-specific models (in the first figure)? Or did the authors count the amount
by sinuhe69 3y ago
Why the amount of the training data for LLM is less than for the distilled and task-specific models (in the first figure)?
Or did the authors count the amount of training data for the LLMs to the required training data for the destined/task-specific models?
https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeIs4yaBA3Ir55j869FMzdmRdf7OxiIjsWl05GU48ikYOHZGLk1H8tIHeKKBaY_xER0QITv5DUhADZvqS1os6mNA_nLQKqwW7DOXnwcnPl6BhsMJ_LKTvglGUrHR5_QC8MIe3K7i9zyfcWkwzvjPhXLifYijgkeeG_1yn9EMm-ol9eI9Cv_rz71wMyGfk2/s1570/image3.png https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj...
- swsieber 3y ago> Or did the authors count the amount of training data for the LLMs to the required training data for the destined/task-specific models? Yes. They are counting the amount of data you need to collect to solve your problem. I can grab a pretrained LLM, and the data I have to collect in that instance is what I need to fine tune it.